Hobbyists have discovered how to insert custom fonts into AI-generated images

Last week, a hobbyist tested the new Flux AI image collection model was discovered It’s unexpectedly good to create custom-trained typewriters. While more efficient methods of displaying computer fonts have been around for decades, the new technique is useful for AI image enthusiasts because Flux can provide accurate depictions of text, and users can now insert words rendered in custom fonts directly into AI image generations.

We’ve had the technology to accurately create smooth computer-rendered fonts in custom shapes since the 1980s (1970s in research), so creating an AI-replicated font isn’t big news. But a new technique means you can see a particular font appear in AI-generated images, like a chalkboard menu in a photoshop restaurant or a printed business card of a cyborg fox.

Shortly after the emergence of mainstream AI image synthesis models such as the standard spread in 2022, some started I wonder: How do I insert my own product, clothing, character or style into an AI-generated image? One answer that emerged came in the form of Lora (a low-grade adaptation). was discovered 2021 allows users to augment knowledge on the AI ​​base model with custom-trained modular add-ons.

These LoRAs, called modules, allow image set models to generate new concepts that were not originally seen (or were poorly represented) in the training data of the base model. In practice, collage hobbyists use them to provide individual styles (say all that is Chalk art) or subjects (detailed pictures Spider-Manfor example). Each lora must be specially trained using examples provided by the user.

Until Flux, most AI image generators didn’t excel at rendering accurate text within a scene. If you were to tempt Standard Spread 1.5 to give you an identity as “cheese”, it would be silly. OpenAI’s DALL-E 3, released last year, was the first mainstream model to do text well. Flux still sometimes makes mistakes with words and characters, but it’s the most adept AI model at rendering “text in the world” (as you might call it) that we’ve seen so far.

Since Flux is an open model available for download and fine tuning, it made sense to train the font LoRA for the first time last month. That’s one An AI enthusiast named Vadim Fedenko (who did not respond to a request for an interview as of press time) recently found out. “I’m very impressed with how it turned out,” Fedenko wrote Reddit post. “Flux takes what characters look like in a certain style/font, making it possible to train Loras with specific fonts, typefaces, etc. Soon to train many more of them.”

For his first experiment, Fedenko chose a bubble “Y2K” style font Reminiscent of what was popular in the late 1990s and early 2000s, the result was released on the Civitai platform on August 20. Two days later, a second typewriter, LoRA, was released by CVdaily user “AggravatingScree7189”. Cyberpunk 2077 Video game.

“The text was really bad before I thought you could do it,” wrote Reacting to Fedenko’s post on the Y2K font, Reddit user eggs-benedryl. Another Redditor wrote“I didn’t know the Y2K journal was fake until I zoomed in.”

Is it too much?

Zoom in / An example Cyberpunk 2077 LoRA is provided with Flux dev.

It’s true that using a deeply trained imagery neural network to render a plain old font on a plain background is overkill. You wouldn’t want to use this method to replace Adobe Illustrator when designing a document.

“It looks great, but it’s funny to reinvent the idea of ​​fonts as 300MB loras.” wrote A Reddit commenter in a thread Cyberpunk 2077 font.

Generative AI is often criticized for its environmental impact, and this is a valid concern for massive cloud data centers. But when Flux runs natively on an RTX 3060, we see that these fonts can be inserted into AI-generated scenes. Measured (scale-reduced) format (and the full dev model can run on an RTX 3090). That’s about as much power consumption as playing a video game on the same computer. The same goes for LoRA creation: creator Cyberpunk 2077 font Trained LoRA in three hours on a 3090 GPU.

There are ethical issues with using AI image generators, such as how they are trained on data harvested without the consent of the content owner. Although the technology is divisive among some artists, a large community uses it daily Share results online Through social media sites like Reddit, this leads to new applications of such technology.

As of this writing, there are only two custom Flux Typeface LoRAs, but we’ve already heard of plans for people to build more as of this writing. While it is still in its early stages, if AI image synthesis becomes more widely used in the future, the technique of generating font LoRAs will become fundamental. Adobe, with its own image collection models, is likely to look at.