Many artists in Web3 use generative machine learning models to create digital art to sell as NFTs. This often involves re-training a model on a dataset collected by the artist, and publishing the output images on an NFT marketplace. Experimenting with newly-collated datasets and training procedures is time-consuming and expensive, meaning that artists are highly protective of these assets. However, recent private AI and decentralized marketplace (e.g. Ocean Protocol) technologies may enable artists to monetize their datasets and models while maintaining control and privacy. This has the added advantage of unlocking more value for artists and NFT enthusiasts through tokens, liquidity pools and staking.
What are Generative Models?
Generative models are a collection of algorithms that are used to automatically discover and learn the regularities or patterns in training data and generate new data samples that resemble the original dataset. You may have come across articles presenting images of real-looking people who do not actually exist. Also known as deep fakes, the ethical implications of these technologies have been widely discussed. It is, of course, important to ensure that this new technology is used safely. Nonetheless, there are also many good uses of the technology such as creating synthetic data in tasks where not much data is available e.g. GANs for biological image synthesis. Generative models can be image-based (as in the example above), text-based (e.g. language models that predict the next word in a sequence such as GPT-3), or even a combination of both (e.g. text-to-image using VQGAN + CLIP).
Why are Generative Models so popular for Art and NFTs?
Recently, generative models have become very popular for creating digital art with the enhanced capabilities of deep neural networks and Generative Adversarial Networks (GANs). These techniques allow for changing the appearance of images in a controllable manner. Furthermore, these tools have become more accessible to artists through frameworks like ml4a (machine learning for artists). The recent technological innovation provided by NFTs (non-fungible tokens) has opened up endless possibilities for dynamic artistic creations and many popular projects have turned to generative models for their collections (e.g. aiBrains, Terra Obscura and GAN-NFT).
How do artists and creators create generative art today?
Creating appealing artistic images is not simply a case of running a pre-trained machine learning model. Artists often spend time collecting new data and re-training a generative model on their own dataset. I’ve come across artists who walked around cities taking photos of interesting buildings, and others who have written poetry paired with images. For example, Holly Grimm re-trained a generative model on an in-house dataset of satellite images during her time as Artist in Residence at Planet. As well as new datasets, breakthroughs in training techniques are often required to generate art with desirable properties. For example, modifying the input noise tensor is needed to produce images with different structures. Finally, real creativity is required to figure out what to prompt the model for with text-to-image models.
For all of these reasons, artists are highly protective of their datasets, trained algorithms and creative know-how for generating artistic images, which can take a lot of time to acquire. At the moment, most artists train their models off-chain on datasets that are off-chain. Only the outputs of the models are published on-chain as NFTs.
How can we innovate on this approach?
Here we identify several possibilities for augmenting current methods, including creating a data economy around generative models, generating more value through tokens and liquidity pools for generative models and using new Web3 technologies in the creation of art.
A Data (and Algorithm) Economy for Artists
There are multiple potential benefits to creating a data economy around the datasets and models of artists. If we were able to make certain guarantees to artists on maintaining control and privacy of assets, it could facilitate interesting collaborations between artists on different combinations of datasets and models. With Ocean Protocol for example, artists can bring access to art datasets and generative algorithms on-chain while maintaining control and privacy using private AI technology (such as Compute-to-Data). Rather than sharing a download link to data, the artist would permit another artist’s algorithm to come to the location of their dataset and run some training code, before returning the trained model.
Tokens and Liquidity Pools for Generative Models
Using a Web3 marketplace like Ocean also has the potential to generate more value for artists and NFT enthusiasts. Currently, artists and NFT enthusiasts are restricted to single action in the market, i.e. buying/selling the output of a generative model. By publishing a generative art algorithm model on the Ocean marketplace, the artist mints tokens for the algorithm and creates a liquidity pool to exchange Ocean tokens for algorithm tokens. If an NFT enthusiast is convinced that a particular NFT collection will become more popular, they can buy tokens for the algorithm used for this collection in the expectation that the value of the tokens will increase in future. Furthermore, they can provide liquidity to the pool and earn yield for algorithms that they think will be regularly consumed.
New Web3 Technologies for Creating Art
Artists have previously incorporated the idea of crowdsourcing into their art pieces. One example uses the Web2 platform Amazon Mechanical Turk to crowdsource a single line from each individual (drawn with a computer mouse) that is then combined into the final piece. With Web3, we could create data unions around artistic datasets that are used to create a work that is owned by the contributors. For example, each individual could upload a single image of a dataset, that is used to re-train a generative model and output images.
Another possibility with Web3 is to create autonomous artificial artists that generate art based on economic and social interactions with humans. The Abraham.ai project are currently working on building out this technology. These artists can be thought of as AI Art DAOs that aim to maximise their treasury by modifying the internal parameters of their generative model based on their success with selling output images.
How have Algovera been working towards this?
With Algovera, we’ve been working on a Proof of Concept for bringing access to generative models on-chain using the Ocean marketplace. We just published an algorithm for training a simple generative model on a private dataset using Compute-to-Data. This algorithm was developed by a decentralized group of 21 individuals that includes data scientists and artists during our weekly hacking sessions (check out the YouTube recordings here). Our mission is to start a creator economy around data science and AI applications and work towards decentralizing AI. We’re passionate that AI models should be owned by a community of individuals rather than one centralized party. To explore new ownership models for algorithms, we launched a DAO and distributed the membership tokens equally among contributors. The role of this DAO is to govern the model and control the flow of any value generated. Our next round of hacking sessions will focus on creating a web app to serve the model and generate revenue for the DAO. If that sounds interesting, make sure to join our Discord and get involved. Alternatively, if you’re interested in collaborating on art projects using technology in our stack, get in touch!