…and robots sang in Abbey Road…

How can AI and machine learning be used in the creative industry?

Making art is a fundamentally human endeavour. We like to notice the little imperfections that show a real person created something: the brush strokes on a Van Gogh canvas, the duff notes in a Beatles record, the fingerprints on Wallace and Gromit puppets.  It makes us feel connected to the creation in front of us, in a very – well, human, way.  

So how can something as cold and soulless as artificial intelligence (AI) be at all useful for artists and creators? It’s hard to picture ‘Jaws’ being half the classic it became if Steven Spielberg had merely written a plot outline and sent it to an AI generator to compose the rest of the script, for instance. There are some subtleties and nuances that surely only the human brain can capture, right? 

Well, yes. That’s perfectly true. And that’s also the point. If you’re looking to use AI as a means to replace human creatives, then it just isn’t going to work. At this stage, the technology is simply no substitute. 

But if you approach AI-powered software as simply as another tool in the creator’s kitbag, then suddenly its use becomes all the more obvious. There is an ever-growing supply of specialised apps, websites and plugins, all serving useful purposes and delivering increasingly impressive results.  Here are just a handful of ways in which AI can be used to great effect during the creative process.  

Using AI to restore and upscale film

Peter Jackson’s Get Back documentary, which premiered last autumn on Disney+, was met with rave reviews from both Beatles fanatics and film fans alike. And while much of its success can be attributed to the mere presence of John, Paul, Ringo and George, it’s also fair to say that Jackson’s visual and audio restoration techniques turned many a head. How did he pull this off? With a lot of hard graft – and the helping hand of AI and machine learning. 

Jackson’s task was to make the raw footage originally shot for the 1970 film Let It Be look at home on 21st century streaming platforms. This was quite a daunting proposition: shot on 16mm film, the 1970 footage was grainy, jerky and muddy. Let It Be has a reputation for being a depressing documentary, and it’s fair to say that the muted colour palette of the footage contributes to this. 

With the help of AI, Jackson was able to restore the tapes to their full potential. The footage was upscaled using AI, sharpening the grainy footage into magnificent HD quality. Colours were now vibrant and inviting, accurately reflecting the playful styles of the late 1960s. Even the audio was improved with AI techniques, which helped separate mono recordings into separate stems for ease of mixing (more on this later!). 

Get Back was always likely to be a success. But Jackson’s clever use of AI restoration techniques breathed so much life into footage that had previously appeared dour and downtrodden. Where Let It Be felt miserable, Get Back is much more celebratory and uplifting in tone. Seeing the 1960s with this clarity helped millions of viewers feel like they had been transported back in time. And the AI-enhanced visuals undoubtedly play a vital role in this effect. 

Using AI to separate stems in songs and audio

Maybe you’re a DJ looking to find ways to transition smoothly between tracks in a live set. Maybe you’re a producer who doesn’t like the way a song is mixed, and fancy having a crack yourself. Or maybe you’re a budding bassist who wants to learn all the intricacies of a track without the distraction of the other instruments in the mix.  

In all these situations, having access to a song’s multitracks and stems is at the very least helpful, and in some cases essential. But these aren’t readily available for the vast majority of tracks out there.  

Enter splitter.ai. An innovative AI-led engine, splitter does pretty much what it says on the tin. Upload a song, and it will generate isolated stems for up to five instruments completely free. Through machine learning, splitter recognises the sound of different instruments, and is able to isolate them from the rest of the track.  

This system can also be used for other types of audio, as Peter Jackson (that man again!) demonstrated so effectively in Get Back. The raw footage contains several scenes where the Beatles’ members would strum their guitars loudly to obscure conversations they didn’t want to feature in the Let It Be movie. Jackson and his team developed a custom-built AI interface to isolate the dialogue and remove the guitar strumming, allowing audiences to listen in to fascinatingly intimate moments the band had wanted to keep between themselves all those years ago. The morality of this is slightly dubious, as Jackson himself admits, but the results are incredibly compelling. Jackson refers to this process as ‘demixing’ – a term that we’ll surely be hearing a lot more of in the future. 

Using AI to create viral memes

AI is by no means perfect yet. Weirdly, that’s almost become one of its strengths recently. Certainly, it’s been a huge factor in how AI-generated imagery has become one of the memes of the moment.  

If you’re on Twitter, chances are you’ve seen some downright bizarre AI-created mashups recently. ‘Gregg Wallace winning Wimbledon’, ‘Tony Soprano playing cricket’, ‘80s horror movie about a priest’s killer hamster’, that kind of thing. While the prompts are often so incongruent they could only be the work of a human brain, craiyon (formerly DALL-e mini) is the AI engine that generates the often-horrifying visual accompaniments. The responses don’t always nail the brief – results can often be quite ghastly - but that’s why it’s funny.  

Gen-Z is often drawn towards the outlandish and the inexplicable, and that’s exactly what craiyon produces regularly. The imperfection and unpredictability of the AI model is what makes it so oddly compelling to an audience who have grown to love the ‘deep-fried’ nature of memes.  

Using AI to identify fonts

Any graphic designer will know the pain of trying to replicate a font they don’t know the name of. Thankfully, machine learning has developed to the point where AI-based systems can identify fonts from almost any image. Websites such as WhatFontIs.com allow users to upload images and get a quick identification of whatever font is present. Their system considers variables such as awkward angles and pixelated image quality to present as accurate a recognition service as is possible right now. It’s not a failsafe method just yet, but it certainly makes the process a lot easier than it used to be.

The future of AI in the creative industry

Machine learning and AI technology has come an incredibly long way in the last few years. There’s a very high chance you’ve been fooled by an AI-generated deep fake at some point recently without even realising it, such is the quality. But it still has a long way to go – we’re nowhere near seeing the full potential of AI at this point. Where the technology ends up is anyone’s guess. But there’s no doubt that, if used as one of many tools, AI has an important role to play on any creator’s workbench. With the human placed at the centre of the creative process, AI can be used to great effect.