Image generation in your product: architecture, safety, cost
A generate button is a job system with a model at the end of it. Start with a hosted API behind your own interface, run every request as an asynchronous job, moderate both the prompt and the picture, keep provenance metadata intact all the way to the download, and give each user a spending limit from the first release. The model call is the easy part: the queue, the policy and the cost controls are the product.
What you are really building
On a whiteboard the feature is one arrow: prompt in, image out. In production it is a pipeline with eight or nine stages.
A request arrives. You check who is asking, whether they have credit and what they asked for. You turn their few words into a prompt the model handles well, queue the job, call a provider and handle the failure when it comes. You check what came back, store it with a record of how it was made, and deliver it.
We will follow one example throughout: a design tool for small businesses, with 20,000 monthly users, adding “generate a background image for this social post”. Nothing about it is exotic, and that is the point.
Hosted API or open-weight model?
| Hosted API | Open weights on your own GPUs | Open weights on an inference platform | |
|---|---|---|---|
| Time to a first version | Days | Weeks | Days |
| Cost shape | Per image, scales to zero | Per GPU hour, busy or idle | Per second or per image |
| Content filter | Provided, with few settings | Yours to build | Usually yours to build |
| Custom style training | Limited or none | Full | Usually supported |
| Where images are processed | The vendor’s infrastructure | Yours | The platform’s |
| Operations | Rate limits and version changes | Drivers, scaling, queues, model updates | Little |
For the design tool we would start hosted. Usage of a new feature is spiky and hard to forecast, and an idle GPU costs the same as a busy one. The reasons to move later are specific: steady high volume that makes reserved hardware cheaper, customer contracts that forbid sending uploads to a third party, or a house style that needs a trained adapter.
Two cautions on open weights. “Open” describes access to the files, not permission to sell with them. Black Forest Labs, for example, publishes open-weight FLUX models, and its licensing page says a commercial licence is needed if you charge for products made with the models listed there. Read the licence for the exact checkpoint. And a self-hosted model usually arrives without the safety filtering and provenance marking that a hosted API applies for you. Both become your job.
Whichever you choose, put the provider behind an interface of your own. Model versions change several times a year, and you will want to swap.
Queues, progress and delivery
Every generation is a job
Image generation is slow by web standards. OpenAI’s image generation guide says complex prompts may take up to two minutes. A request that long will hit a timeout in a load balancer, on a mobile network or in the user’s patience.
So the API accepts the request, writes a job row and returns its ID at once. A worker picks the job up. The client polls, or listens on a server-sent event stream.
{
"job_id": "gen_01JCX4",
"status": "generating",
"request": { "template": "social-bg-v4", "size": "1024x1024", "quality": "draft" },
"provider": { "name": "hosted-a", "model": "image-model@2026-10", "request_id": "..." },
"idempotency_key": "c1f0...",
"credits_reserved": 1
}
The status moves through queued, moderating_input, generating, moderating_output and ready, or stops at blocked or failed. The details that save trouble later:
- Idempotency. A double click or a retried request must not create, or bill, two jobs.
- Provider callbacks. Some providers have you poll and others call a webhook. Black Forest Labs’ integration guidelines require polling the URL returned with each request, and allow 24 concurrent requests on most endpoints. Platforms such as Replicate post to a webhook when a prediction changes state. Verify webhook signatures, and treat a callback as a hint to fetch the result, not as the result.
- Your queue is the buffer. Provider limits apply to your account, not to each of your users. When 300 people press the button at nine on a Monday, the queue absorbs it and the interface shows a position. The patterns are the same as in scaling an LLM application.
- Progress. Show stages, not a fake percentage. Where the provider streams partial images, show them: OpenAI’s API can send up to three partial frames.
- Failure. Decide what a timeout means. Retry once with backoff, then mark the job failed and release the reserved credit. Users forgive a failure that costs them nothing.
Storage and CDN
Do not serve images from the provider’s URL. The same Black Forest Labs guidelines say result URLs expire after ten minutes and are not meant for serving to end users. Download the file inside the worker, write it to your own object storage and serve it through your CDN.
Keep originals private and hand out signed URLs, because generated images often contain a user’s uploaded logo or product. Store the prompt, the template version, the model version, the moderation results and the provenance data next to the file. Decide retention before launch. When a user deletes an image, delete the derivatives and purge the CDN too.
None of this is unusual cloud architecture, but it is easy to put off.
Prompt help, templates and reference images
Most users type four words and are disappointed. The quality of the feature depends more on what you do with those four words than on which model you call.
Templates. In the design tool, the user picks “background for a social post”, a mood and a colour, and types a subject. The server assembles the full prompt from a versioned template that carries the things users never think to say: leave the centre calm for a headline, no text in the image, the aspect ratio of the post. Templates are product assets. Test them like code.
Rewriting. A small language model can expand a short request into a detailed description. Show the user the expanded prompt so they can correct it, and moderate both versions, because rewriting can turn a bad request into an innocent-looking one.
Reference images. Current hosted models accept several reference images in one request, which is how you keep a brand’s look from one generation to the next. Let a workspace save a style and attach it automatically. Uploads may contain people and may not belong to the uploader, so scan them as carefully as outputs.
Text. Do not have the model draw the headline. Generate the picture and set the type in your own editor, with real fonts. Our guide to AI image generation for business covers what models can and cannot draw.
Moderation, abuse and rate limits
Assume that within a day of launch someone will try to make something you would not want your name on.
On the way in. Check the text prompt against your own policy before spending money on generation. The policy is yours to write: sexual content, real people by name, children, violence, other companies’ brands and characters. Check uploaded images too. Provider filters help but are not the whole answer. OpenAI’s image API, for example, exposes a moderation setting with two values, auto and a less restrictive low. Your rules will be more specific than any vendor default.
On the way out. A harmless prompt can produce a harmful picture, so classify every output before the user sees it. A vision language model with your policy written out does this adequately for most categories, and the trade-offs are in vision language models vs classic computer vision. Known illegal content needs a specialist detection service and a legal reporting process, arranged before launch and not after.
When you block. Tell the user plainly that the request was refused, do not charge for it, log it, and offer a way to appeal. Count refusals per account. Three in ten minutes is a pattern.
The law has moved here as well. The EU AI Act’s implementation timeline adds prohibitions from 2 December 2026 covering AI systems that generate non-consensual sexual deepfakes and child sexual abuse material. Any product that generates images of people needs safeguards it can document.
Rate limits. Free generation attracts automated sign-ups. Require a verified email before the first job, limit each user to one or two concurrent jobs, cap jobs per hour, and watch for many accounts behind one device or payment card.
Do AI-generated images need labelling?
Increasingly, yes, and in two layers: a mark that machines can read and a notice that people can see. What follows is a planning summary, not legal advice.
The EU rule. Article 50 of the AI Act has applied since 2 August 2026. Providers of AI systems that generate synthetic images must ensure the outputs are marked in a machine-readable format and are detectable as artificially generated. Systems placed on the market before that date have until 2 December 2026 to meet the marking duty. Separately, whoever deploys a system to create a deepfake must disclose that the content is artificial. There is an exemption for systems that perform an assistive function for standard editing or do not substantially alter the input, and it is narrower than product teams tend to hope.
The question to settle with a lawyer is your role. If you put a generation feature on the market under your own name, you may be the provider of that AI system even though someone else trained the model. Plan as though the marking duty reaches you. The Commission’s voluntary code of practice, published in June 2026, describes ways to meet it, and our article on the EU AI Act for product teams covers the rest. Other jurisdictions have their own disclosure rules, so check each market you sell into.
The technical means. The main standard is C2PA Content Credentials: signed metadata attached to a file that records where it came from and what was done to it. The standard’s steering committee includes Adobe, Google, Microsoft, OpenAI and Sony. OpenAI’s content provenance guide describes two signals in its images, signed C2PA metadata and a SynthID watermark embedded in the pixels, and Google’s documentation says all images from its Gemini image models carry SynthID.
The trap is your own pipeline. Signed metadata does not survive a careless resize, a format conversion or an image-optimising CDN. OpenAI’s guide says as much: editing or sharing a file can remove the metadata, while the watermark may survive some transformations. So:
- Test what leaves your system, not what enters it. Download an image the way a user would and inspect it with a verifier.
- If you transform images, preserve the manifest or sign the result again with your own credentials, recording the edit.
- Keep a server-side record linking each file hash to its job, as a fallback when metadata is stripped downstream.
- Label generated images visibly in your interface and in the share and export flows.
What does an image cost, and how do you cap it?
The fee per image is the visible part. The full cost of the feature includes every image the user generated and threw away, moderation calls on both sides, the prompt-rewriting call, storage and delivery, and failed jobs the provider still charged for.
The number to manage is cost per kept image: total spend divided by the images users went on to download, publish or place in a design. If users generate eight images to keep one, the true unit cost is eight times the list price, plus overheads.
Settings change the bill more than model choice does. In OpenAI’s published token table for its earlier GPT Image models, a high-quality square image uses about fifteen times the output tokens of a low-quality one. That ratio suggests the design:
- Draft first. Generate small, low-quality previews. Render the full-quality version only for the one the user picks.
- Credits per user. A monthly allowance tied to the plan, visible in the interface, reserved when a job starts and released if it fails.
- A service-wide daily budget. When spend reaches it, generation pauses for free users and someone is paged. An attack or a bug then costs one day’s budget at most.
- Alerts on the keep rate. A falling keep rate means prompts or templates have got worse, and it shows before the invoice does.
How to start
In the first week, build the skeleton with one hosted model: the job table and queue, the interface around the provider, moderation in both directions, storage in your own bucket and a credit counter. Release it to staff only. Then download a generated image as a user would and check that its provenance metadata is still there.
The first decision is not the model. It is the policy: what users may generate, what you refuse, and who reviews an appeal. Write that page before the prompt template, because moderation, labelling and the limits on uploads all follow from it. Model choice can change in an afternoon if the interface is thin. Policy mistakes are public.
Building generation features with the queueing, safety and cost controls in place from the start is part of our generative AI work.