Skip to content
sysout.dev

In My Head — Model Polish Strategy

Decision summary

Model polish should be a controlled transformation of a transcript, not one generic rewrite action. The user should choose a polish profile: a named, composed set of policies for audience, change budget, tone, format, and fidelity. The user does not need to assemble those primitives for the common case. IMH supplies useful built-in profiles, while advanced users can adjust the primitives or save a custom profile.

The profile is the user-facing choice. Audience, change budget, tone, and format remain the internal primitives because they can be reused across many profiles and tested independently. Terms such as summarise, rewrite, and adapt describe what a profile may produce, but should not become the main menu of disconnected operations.

The source transcript is always retained. Every accepted model result is a new version with its strategy recorded. Automatic polish is allowed to use only the lowest-change strategy. Higher-change strategies are manual and reviewable.

This separates two risks that the current single prompt mixes together:

  • Fidelity risk: changing facts, names, numbers, technical terms, or the speaker’s intended meaning.

  • Authorship risk: producing wording that no longer feels like the user’s own voice.

The source is always a voice-capture transcript. Source context is therefore not a user-facing profile choice. The source version still matters for versioning, comparison, and audit.

The immediate experiment is narrower: manual polish temporarily omits the default cleanup system prompt and sends the optional instruction with the transcript as the user message. Automatic polish remains conservative. This experiment is useful for comparing model behavior, but it is not the final profile or prompt API.

Functional analysis

What polish is for

An IMH transcript can be useful in several different states. A person may be keeping a private thought, preparing work notes, drafting a blog post, or creating content for someone else to use. These are different jobs with different acceptable amounts of model invention, omission, structure, and voice change.

Those jobs should be represented by profiles rather than by a flat list of model operations. For example, Clean note can mean conservative cleanup, while Article draft can combine a broader audience, a larger change budget, an appropriate tone, and an article format. The primitives remain visible in the profile’s edit surface and in the result contract, but they do not need to be the first decision every time.

Audience and change budget

Audience is about the people or consumer for whom the result is prepared. It is not a list of professions. An executive, doctor, lawyer, or teacher can use the same audience value with a different format, tone, or domain-specific profile.

The initial user-facing audience values should be:

  • Me — a private note for the person who captured it.

  • Friends and family — a personal reader who benefits from natural, conversational language.

  • People I work with — colleagues or collaborators; preserve decisions, owners, dates, and work-specific terms.

  • Broader audience — readers who may not know the background. This can cover a public post, executives, or another department, but it does not by itself decide whether the result should be a post, brief, or internal update.

  • Experts — readers who can handle domain terminology and detail. The model should preserve precise terms rather than simplify them aggressively.

  • Another tool — output intended to be copied into a tool or downstream model; use explicit structure and stable terminology.

The audience changes language and emphasis; the change budget controls how much of the original wording and structure may move. A profile combines both.

Audience Change budget Appropriate result Must preserve Confirmation

Me

Preserve

Punctuation, paragraphs, obvious fillers, false starts

Facts, uncertainty, personal voice, level of detail

Automatic or one-tap manual

Me

Improve

Clearer sentences and useful headings

Meaning, named entities, numbers, chronology, uncertainty

Review suggested version

Friends and family

Improve or reorganize

Readable explanation or short message in a natural tone

Meaning and important context; no invented background

Review before accept

People I work with

Improve or reorganize

Concise notes, decisions, code-adjacent explanation, action items

Technical identifiers, commands, APIs, numbers, owners, dates

Review with protected-facts warnings

Broader audience

Adapt

Professional, neutral, audience-aware communication

Explicit facts and claims; uncertainty must remain visible

Explicit confirmation required

Another tool

Transform

Structured output for a requested tool or downstream workflow

Only facts selected by the requested format; no unsupported additions

Explicit confirmation and separate output version

Experts can use Preserve, Improve, or Transform depending on the intended format. It is not inherently a high-change audience. Likewise, Broader audience does not mean “public” and should not force a public-writing tone.

“Preserve”, “Improve”, “Reorganize”, “Adapt”, and “Transform” are discrete change-budget policies, not necessarily user-facing operations. They give the model and the review surface a legible contract without making the first-run experience a settings dashboard.

Profiles

A PolishProfile is the composed entity that users select, edit, and save. It is a better long-term name than style: style usually suggests tone only, whereas a profile includes the intended reader, allowed change, format, and fidelity rules. It is also more durable than destination, which sounds like an export location.

Each profile contains:

  • a user-facing name and stable profile ID;

  • audience policy;

  • change budget;

  • tone and vocabulary guidance;

  • output format;

  • protected-content and uncertainty policy;

  • an optional custom instruction; and

  • whether it is available for automatic polish.

The default profile is Clean note:

  • Audience: Me.

  • Change budget: Preserve.

  • Tone: neutral and recognisably the speaker’s.

  • Format: plain text with natural paragraphs.

  • Policy: remove spoken artefacts and repair punctuation, but do not add, remove, summarize, or reinterpret content.

  • Automatic: allowed.

This default should make the first result useful without requiring a choice. Manual polish can start from the same profile and let the user adjust it. Useful built-in profiles can then include Clear explanation, Work note, Article draft, Shareable message, and For another tool. These are starter recipes, not an attempt to enumerate every profession or use case.

Recommended user experience

The current polish sheet can evolve into a small profile sheet without turning the transcript screen into a settings dashboard.

  1. Keep Run polish as the primary action.

  2. Show IMH profiles first, with Clean note selected by default.

  3. Let Adjust profile expose the primitives as visible chips: audience, change budget, tone, and format. Use the six plain-language audience values above so the composed choice can be understood at a glance.

  4. Keep an optional free-form instruction visible for every built-in profile. It supplements the selected profile and cannot silently weaken its fidelity contract.

  5. Provide a prompt-details affordance for the explicit manual run. Show the equivalent request as profile guidance, optional instruction, and a transcript placeholder. Do not mix the automatic polish prompt into this preview.

  6. Under the controls, show a short contract such as “Keeps facts and your uncertainty; may change sentence structure and headings.”

  7. For the default and other low-change profiles, run directly and show the version comparison. For profiles with a larger change budget, require the user to confirm the higher-change contract before generation.

  8. Present the result as a new version with Keep original, Accept result, and Try again with a different strategy. Never overwrite the raw or previously accepted version.

The comparison view should make the level of intervention visible. A compact summary can report “Changed wording”, “Added structure”, “Removed content”, and “Protected facts differ”. The user does not need a claim that the model was correct; they need help deciding whether this result is acceptable.

Automatic polish

Automatic polish remains deliberately narrow:

  • Profile: Clean note.

  • Audience: Me.

  • Change budget: Preserve.

  • No user-supplied style or audience instruction.

  • Raw Whisper text is persisted before the model runs.

  • A failed automatic pass leaves the transcription successful and exposes a polish-specific retry.

Automatic polish should not become an implicit place where the user has to discover that the model rewrote their voice. If a future setting allows a stronger automatic mode, it should be a separate, clearly named opt-in.

Prompt and model policy

Prompt layers

The prompt builder should be explicit about layers instead of concatenating one large fixed prompt with an “additional instruction” suffix.

PolishRequest resolves to a PromptPlan containing:

  • a system policy for the selected profile;

  • an audience policy, when applicable;

  • a change-budget policy;

  • tone and vocabulary guidance;

  • an output-format contract;

  • the user instruction, if any; and

  • the transcript payload.

The system policy defines the non-negotiable safety and fidelity contract. The user instruction expresses intent. The transcript is clearly delimited as source material. This makes it possible to test each part independently and prevents a future style template from accidentally weakening fact preservation.

Audience-to-prompt mapping

The prompt builder should compile semantic policy into positive instructions. It should never emit internal values such as Unknown, None, or Unspecified as if they were audience labels.

The initial mapping is:

User value Prompt behaviour

Me

Preserve the speaker’s voice and personal shorthand where it remains readable. Do not explain private context or make the note public-facing.

Friends and family

Use natural, conversational language. Explain specialised references only when the profile or source makes that useful; do not invent background.

People I work with

Prefer clear, concise language. Preserve decisions, owners, dates, identifiers, and work-specific terms.

Broader audience

Make the result understandable without assuming shared background. Expand or explain important terms when supported by the source; do not infer facts.

Experts

Preserve domain terminology, precision, and useful detail. Do not simplify away distinctions or add expert conclusions.

Another tool

Use explicit structure, stable terminology, and low conversational noise. Follow the selected format or schema; do not invent fields or values.

Audience is only one layer. For example, an Executive brief profile may use Broader audience plus a concise brief format and a larger change budget. A Department update may use People I work with plus “assume no shared background” and a structured update format. A Public post may use Broader audience plus a shareable format. This is why public, executive, and department-specific values should not become a growing audience enum.

The internal policy needs to distinguish absence of information from an explicit constraint:

Internal state Prompt construction

Audience unspecified

Omit audience-specific instructions and use the default fidelity and readability policy.

Reader familiarity unknown

Do not assume expertise. Prefer clear language without aggressive simplification or invented explanations.

Shared context absent

Make the result self-contained, preserve ambiguity, and do not invent the missing background.

Tone or format unspecified

Omit that layer and retain the profile’s default neutral tone and plain text format.

None means an explicit “there is no shared context” policy; Unknown means IMH does not know. Both are internal states and normally collapse to positive prompt clauses or omission. They should not be shown as user choices.

The renderer should assemble the final request in a stable order:

  1. non-negotiable fidelity and safety policy;

  2. change-budget contract;

  3. audience and shared-context policy;

  4. tone and format contract;

  5. optional custom instruction; and

  6. clearly delimited transcript source.

The custom instruction can refine the selected profile, such as “use a warm tone,” but cannot silently override its protected-content or change-budget rules. Each layer should have unit tests that verify both the rendered prompt and important omissions, such as no audience clause for an unspecified audience.

Strategy contracts

The initial contracts should be short and testable:

  • Preserve: fix transcription artefacts and punctuation; do not add, remove, reorder, summarize, or reinterpret content.

  • Improve: clarify grammar and sentence boundaries; preserve every fact, qualifier, technical term, and level of detail.

  • Reorganize: improve headings, paragraphs, bullets, and ordering only when the source supports that structure; preserve all substantive content.

  • Adapt: write for the selected audience and tone; preserve claims, uncertainty, names, dates, numbers, and technical identifiers.

  • Transform: follow the requested format and explicitly allow omission or compression; do not invent facts, decisions, owners, dates, or actions.

These are policies, not guarantees. The UI must still present the result as a model suggestion requiring review.

Profile templates

The future profile roadmap should use named templates with visible contracts, not opaque prompt strings. A profile contains:

  • name and stable ID;

  • audience and shared-context policy;

  • allowed change budget;

  • tone and vocabulary guidance;

  • structure or format rules;

  • protected-content policy; and

  • whether it is available for automatic polish, manual polish, or transform only.

Examples include Formal, Informal, Developer, Meeting notes, and Plain language. “Developer” should mean technically precise and identifier-preserving; it should not mean adding code or guessing missing implementation details.

Technical design for IMH

Domain model

Introduce IMH-owned profile and request models rather than passing nullable strings through the service:

data class PolishProfile(
    val id: String,
    val name: String,
    val audience: PolishAudience,
    val changeBudget: PolishChangeBudget,
    val sharedContext: SharedContext = SharedContext.Unspecified,
    val tone: PolishTone = PolishTone.Neutral,
    val format: PolishOutputFormat = PolishOutputFormat.PlainText,
    val protectedContentPolicy: ProtectedContentPolicy,
    val customInstruction: String? = null,
    val automaticAllowed: Boolean = false,
)

data class PolishRequest(
    val sourceVersionId: String,
    val profileId: String = "clean-note-v1",
    val customInstruction: String? = null,
    val automatic: Boolean = false,
)

PolishAudience, PolishChangeBudget, PolishTone, PolishOutputFormat, and ProtectedContentPolicy should be closed enums or sealed values. Built-in profiles should have stable versioned IDs such as clean-note-v1; user-created profiles need their own persisted IDs and snapshot metadata. The service should resolve the profile ID to a versioned PromptPlan; persisted versions should store the stable profile ID and prompt-plan version, not the full prompt text alone.

The current TranscriptVersion.customInstruction remains useful for display and audit. It should be supplemented by strategy metadata rather than made to carry the entire policy.

Service flow

The target flow is:

  1. Load the selected source version.

  2. Resolve and validate PolishRequest; reject an automatic request whose profile is not marked automaticAllowed.

  3. Extract protected facts from the source for diagnostics: numbers, dates, URLs, code-like tokens, and explicit names. This is not a replacement for human review.

  4. Build a PromptPlan and record a request-start diagnostic.

  5. Generate using the local model under the existing model mutex.

  6. Validate non-empty output, output limits, and protected-fact drift.

  7. Save a new transcript version only after validation. The source remains active until the new version is committed.

  8. Show the diff and let the user accept or keep the source.

The current manual no-system-prompt experiment can be represented as a temporary PromptPlan with an empty default system policy. It should be removed or converted into an explicit strategy once the A/B result is known.

Persistence and status

Every model result needs enough metadata to answer “what changed this?”:

  • profile ID, audience, change budget, tone, and output format;

  • custom instruction;

  • prompt-plan version and model identifier;

  • backend selected and generation configuration;

  • source version ID; and

  • validation outcome, including protected-fact drift if detected.

Transcription and polish remain separate state machines. A successful Whisper transcription must never become a transcription failure because a model polish pass failed. A polish failure should retain the source version, expose the error, and allow retry with another strategy.

Long transcripts

The model context and output budget must be treated separately. For a long transcript:

  1. Estimate input and desired output tokens before generation.

  2. Split at paragraph, sentence, or source-version boundaries with a small overlap only when needed for continuity.

  3. Apply the same PromptPlan to each chunk.

  4. Preserve chunk order and run a lightweight stitch pass only when the total result fits the available context; otherwise concatenate with deterministic paragraph joining.

  5. Record that the result was chunked and retain chunk-level diagnostics.

  6. Never silently truncate the source or pretend a partial result is complete.

The runtime adapter should eventually expose a structured generation result with completion reason and token counts. A String? result cannot distinguish normal completion, empty output, cancellation, and output-limit termination.

Backend and UX diagnostics

Backend selection belongs to the model/runtime boundary, but its user-facing state belongs to IMH. The polish progress surface should eventually say Preparing model, Polishing on GPU, Polishing on CPU fallback, or Polish failed. Logcat remains useful for diagnostics, but should not be the only way to know which backend produced a result. IMH now receives the successful backend from model-core and exposes Preparing model, Polishing on GPU, and Polishing on CPU fallback in the active polish surface.

Testing and evaluation

Create a small fixed corpus before selecting a final default prompt:

  • private diary/thought capture;

  • meeting discussion with decisions and action items;

  • developer notes containing APIs, identifiers, and code fragments;

  • informal message; and

  • a transcript with uncertainty, numbers, names, and corrections.

For every corpus item, compare the strategies against the same model and runtime configuration. Measure:

  • protected-fact preservation;

  • omission and addition rate;

  • change in length and structure;

  • user-rated voice preservation;

  • user-rated usefulness for the selected audience; and

  • acceptance or retry rate.

The first physical-device A/B test should compare only these two inputs:

  • current conservative system prompt; and

  • Gallery-like no-default-system-prompt manual input.

Keep backend, model file, sampling values, transcript, and output budget constant. Otherwise a prompt experiment cannot explain the result.

Recommended delivery order

  1. Finish the current no-default-system-prompt manual experiment and capture paired outputs.

  2. Add profile metadata to model-polished versions without changing the existing UI choices.

  3. Replace the single instruction sheet with built-in profiles and an Adjust profile advanced path.

  4. Add audience, tone, and format choices for manual profiles.

  5. Add protected-fact diagnostics and a stronger diff/review surface.

  6. Add chunked long-transcript handling and structured runtime results.

  7. Add reusable profiles and only then consider stronger automatic polish modes.

Design references

The strategy is informed by current writing-assistant patterns: