Word-As-Image for Semantic Typography

July 20, 2023

In this episode of the Talking Papers Podcast, I hosted Shir Iluz. We had a great chat about her paper “Word-As-Image for Semantic Typography”, SIGGRAPH 2023 Honorable Mention award winner.

This scientific paper presents an innovative approach for text morphing based on semantic context. The authors demonstrate how they transform an input word, such as “bunny,” into a visually striking representation where each letter takes on the appearance of a bunny. To achieve this, they leverage a bezier curve representation with control points, which is subsequently processed through a rasterizer and a vector diffusion model. This optimization-based approach ensures that the semantics of the word are accurately imposed on the letter shapes. Balancing between readability and semantics is a non-trivial task, and the authors address this by formulating multiple loss functions. These loss functions act as “control knobs,” enabling users to fine-tune the final results, either prioritizing enhanced readability or maximizing semantic meaningfulness. The paper’s compelling results are complemented by an impressive demo, which showcases the transformative capabilities of their methodDon’t miss the amazing demo they created.

Shir, a dedicated Masters student at Tel Aviv University, has embarked on a groundbreaking project in collaboration with Yael Vinker, both making equal contributions to this remarkable endeavor. You may know Yael from her prior appearance as a guest on the podcast (CLIPasso episode). Their work carries immense potential, promising to revolutionize the creative processes of artists and designers. Rather than commencing from a traditional blank canvas or plain font, this innovative approach enables individuals to initiate their logo design journey by transforming a word into a captivating image. The implications of this novel technique hold the power to reshape the very workflow of artistic expression, opening up exciting new possibilities for visual communication and design aesthetics.

I am eagerly anticipating the next set of papers she will sketch out (pun intended).

Check out this result for the word “Podcast” with the semantic context of “Headphones”. I like it.

AUTHORS

Shir Iluz, Yael Vinker, Amir Hertz, Daniel Berio, Daniel Cohen-Or, Ariel Shamir

ABSTRACT

Abstraction is at the heart of sketching due to the simple and minimal nature of line drawings. Abstraction entails identifying the essential visual properties of an object or scene, which requires semantic understanding and prior knowledge of high-level concepts. Abstract depictions are therefore

A word-as-image is a semantic typography technique where a word illustration presents a visualization of the meaning of the word, while also preserving its readability. We present a method to create word-as-image illustrations automatically. This task is highly challenging as it requires semantic understanding of the word and a creative idea of where and how to depict these semantics in a visually pleasing and legible manner. We rely on the remarkable ability of recent large pretrained language-vision models to distill textual concepts visually. We target simple, concise, black-and-white designs that convey the semantics clearly. We deliberately do not change the color or texture of the letters and do not use embellishments. Our method optimizes the outline of each letter to convey the desired concept, guided by a pretrained Stable Diffusion model. We incorporate additional loss terms to ensure the legibility of the text and the preservation of the style of the font. We show high quality and engaging results on numerous examples and compare to alternative techniques.

📚VectorFusion