AI goes beyond being a feature in a product; it's a platform that will transform both the process and the outcomes of design.

In this new age of AI, the opportunities are limitless, extending beyond design to encompass the entire product development lifecycle. AI will be pivotal in how teams operate within Figma, enabling them to quickly create initial drafts and efficiently iterate and ship. AI can more seamlessly bridge the gap between intention and action.

However current AI powered design tools face several significant challenges, especially when it comes to meeting the specific needs of professional designers and maintaining the integrity of established design systems. These include:

1. Low Fidelity Outputs: Many AI design tools generate low-fidelity wireframes and basic layouts. Making them less useful for designers who need high-quality, production-ready assets.
2. Lack of Design System Integration: Existing AI tools typically create vectors from scratch making them only fit for early ideation, as they do not adhere to the specific guidelines and components that make up a company's design language.
3. Inconsistent Quality:AI-generated designs often vary in quality and may not always align with the designer's vision or the project's requirements.
   
   This inconsistency can lead to additional work for designers, who must refine and adjust the AI-generated outputs to fit their needs.
4. Limited Customization: Many AI tools offer limited customization options, making it challenging for designers to fine-tune the generated designs to match their specific requirements. This limitation reduces the overall usefulness of these tools in professional settings where bespoke solutions are often necessary.
5. Intellectual property concerns: Designers and companies may be hesitant to use AI tools that require access to their design files and assets, fearing potential misuse or leakage of sensitive information. Recently Figma temporarily disabled its “Make Design” AI feature that was said to be ripping off the designs of Apple’s own Weather app.
6. Poor Contextual Understanding: AI tools may not fully understand the context and purpose behind specific design decisions, leading to outputs that miss the mark in terms of user experience and functionality. This lack of contextual awareness can result in designs that are not user-centered or aligned with the project's goals.
7. Dependency on Generic Models: Many AI design tools rely on generic models that are not tailored to specific industries, brands, or design languages. This reliance on one-size-fits-all solutions can limit the effectiveness of AI-generated designs in specialised contexts.
   

The seed of the idea

At this time, I was grappling with several challenges at work, such as finding ways to encourage my team to prioritise content and adopt the design system. My interest in AI features and experimentation had been growing, although I was initially skeptical about its potential, after the release of GPT-3.5, GPT-4 and custom GPTs, it was clear that the from a content design perspective it was impressive and I saw an opportunity to explore the untapped intersection of design and AI.

To leapfrog the current generative AI design solutions I began Imagining a tool that could create designs using your design system components, tailored to your specific patterns and tone of voice. I started envisioning the thrilling possibilities of integrating OpenAI's GPT-3 with Figma's API to streamline and enhance initial design explorations.
‍
‍

An engineering challenge

The primary challenge from an engineering perspective was to build the logic to create a coherent and functional design from a single shot prompt.

Working within typescript, I made dozens of iterations to address challenges such as accurate prompt interpretation, response parsing, component mapping, dynamic layout management, and image integration, enhancing the design process within Figma.
‍

Creative problem solving

Working within typescript, I made dozens of iterations to address challenges such as accurate prompt interpretation, response parsing, component mapping, dynamic layout management, and image integration, enhancing the design process within Figma.