Code415 Radar

Key ideas

Adversarial benchmark

A test built on purpose to expose a specific weakness, here the habit of guessing from familiar code instead of reading the spec.

Stack based language

A language where operations push and pop values on a shared stack instead of using named expressions, like Forth or PostScript.

Spec following versus pattern matching

An agent that truly follows a spec obeys odd rules it has never seen, while a pattern matcher falls back on what similar languages usually do.

Steps

1. Read Picheta's blog post "I built a programming language that LLMs can't use" first, focusing on the list of traps.
2. Open the dom96/KillSwitch README and the fizzbuzz example in its tests folder to see what real code looks like.
3. Look at the KillSwitch Bench leaderboard and note which models lead and which fall behind.
4. Compare the traps with ordinary agent failures you have seen, such as ignoring a project rule or inventing an API.
5. Self check: why does Picheta say a score of 66.8 still does not show that a model understands KillSwitch?

Steps

1. Clone https://github.com/dom96/KillSwitch and enter the folder.
2. Install the interpreter with cargo install --path . as the README shows.
3. Run the bundled example with killswitch run tests/fizzbuzz.ks, or use cargo run -- run tests/fizzbuzz.ks without installing.
4. Give your own coding agent the spec and ask it to write a hello world program, then run its output through the interpreter and record each error.
5. Run the test suite with cargo test to see which language rules the interpreter enforces, and match them to the agent's mistakes.

Small angles to try

• Try the same prompt with two different models and compare how many runs it takes to get valid code.
• Use the README's cargo run -- anagram testly tsetly helper to explore the identifier letter rules by hand.
• Note that the repo is AGPLv3 and its README forbids AI training on the contents, so keep experiments local.

Key ideas

Prompt cache

A provider stores the processed start of a long prompt so the next request sharing that prefix is cheaper and faster, but only on the same provider account.

Session affinity

Routing every turn of one conversation to the provider that already holds its cache instead of spreading turns across providers.

Conversation fingerprint

Vercel's fallback session ID, a hash of the first system message, the first user message and the tool set, which stays stable as the conversation grows.

Steps

1. Read the Vercel changelog entry "AI Gateway logs now show session affinity" first.
2. Read the session affinity docs page, especially the table of session ID sources and the default pin lifetimes per provider.
3. Read Siegel's post for the case for owning the proxy layer instead of relying on vendor dashboards.
4. List which of your own tools would hide a cache miss today and which would show it.
5. Self check: why can two unrelated scripts that both start with the same templated message end up in one fingerprinted session?

Steps

1. Install the OpenAI SDK for Node, then create a client with baseURL: 'https://ai-gateway.vercel.sh/v1' and a default x-session-id header, as in the session affinity docs.
2. Send a long system prompt above the provider's cache minimum, then send three follow up turns that reuse the same history and the same session ID.
3. Print provider_metadata.gateway.routing from each response and note clientSessionIdSource and affinity.outcome.
4. Open the AI Gateway logs, group by sessions, and compare the cache hit rate shown there with what you printed.
5. Repeat without the header and confirm the session now appears as fingerprinted.

Small angles to try

• Point Claude Code at the gateway with the documented ANTHROPIC_BASE_URL="https://ai-gateway.vercel.sh/claude-code" setting and inspect a real coding session.
• Set sessionAffinity: false in providerOptions.gateway and see whether the outcome changes to skipped_mode_off and the cache hit rate drops.
• Swap models mid session and check that switching back reuses the earlier provider pin.

Key ideas

Memory write path

The step where an agent turns new conversation text into stored facts and decides whether to add, merge or overwrite existing memories.

Active parameters

In a mixture of experts model only a few experts run per token, so serving cost tracks the roughly 3B active parameters rather than the 30B total.

Held out harness

A test setup the model never saw in training, used to check that gains are not just fitting to familiar prompts.

Steps

1. Read the Hugging Face model card first for the results table and the vLLM quickstart.
2. Skim the arXiv abstract of 2610.22196 to see the two training stages.
3. Read the Hy-Memory README sections on add() and search() to see where the writer model plugs in.
4. Compare the per harness LongMemEval scores to see how much the surrounding framework matters.
5. Self check: why is a held out harness score more convincing than the macro average?

Steps

1. Serve the model with vllm serve "tencent/Hy-Memory-30B-A3B", adding --served-model-name hy-memory as the model card's quickstart does.
2. Install the framework with pip install hy-memory==2.0.3.
3. Point the MEMORY_LLM_* settings at your local vLLM endpoint and the MEMORY_EMBEDDER_* and MEMORY_EMBEDDING_DIMS settings at your embedder, following the README.
4. Create a HyMemoryClient(mode="pro"), call add() with a short made up chat where a preference changes partway through, then call search().
5. Check whether the stored memory reflects the updated preference rather than both versions.

Small angles to try

• Run the same chat with a general model as the memory LLM and compare what each one stores.
• Try scene="coding" with a tool call trace and inspect the facts and experiences channels.
• Serve the model with SGLang using the command on the model card and compare startup and latency.

Key ideas

GGUF

The single file model format used by llama.cpp that packs quantized weights with tokenizer and metadata.

2 bit quantization

Storing each weight in about two bits instead of sixteen, which shrinks memory use sharply at some cost in accuracy.

Retention

The compressed model's benchmark score as a share of the full model's score on the same test.

Steps

1. Read the Underdog-Saluki-27B-1.0 model card first, especially the benchmark table and its caveats.
2. Open the base Qwen/Qwen3.8-27B card to see what the full model was built for.
3. Compare tool calling results with the AIME rows to see where compression hurts.
4. Read the Quickstart and note what the --jinja and -c 32768 flags are for in llama-server.
5. Self check: why might a heavily compressed model match or beat the full one on tool calling but lose on competition math?

Steps

1. Download the file with huggingface-cli download ConwayResearch/Underdog-Saluki-27B-1.0 Underdog-Saluki-27B-1.0-IQ2-mix.gguf --local-dir ..
2. Start the server with llama-server -m Underdog-Saluki-27B-1.0-IQ2-mix.gguf --jinja -ngl 99 -fa on -c 32768, the command on the model card.
3. Send a chat request to the local OpenAI compatible endpoint that defines two simple tools, such as a weather lookup and a calculator, and check that the model returns valid tool calls.
4. Ask a multistep math word problem and compare the answer with a larger model you trust.
5. Record tokens per second and memory use so you know whether it fits your machine.

Small angles to try

• Add vision by passing --mmproj mmproj-Underdog-Saluki-27B-1.0-F16.gguf and ask about a local screenshot.
• Lower -c to see how context size changes memory use.
• Plug the local endpoint into a coding agent that supports custom OpenAI compatible providers and try a small refactor.

Key ideas

Grounded generation

Producing output only from documents the user supplies, rather than from the model's general knowledge.

Retrieval practice

Learning by recalling an answer from memory, which short answer and fill in the blank questions demand more than multiple choice.

Source to artifact pipeline

Turning one private set of documents into several formats such as video, slides and quizzes, a common pattern in RAG products.

Steps

1. Read Google's post "three tips to study with Gemini Notebook" first.
2. List the three features and the inputs each one takes.
3. Think about how you would check that a generated quiz uses only facts from the uploaded files.
4. Compare these outputs with a plain chat answer about the same notes.
5. Self check: what makes a fill in the blank question harder to fake than a multiple choice one?

Steps

1. Open notebook.google and create a notebook from your own notes on one technical topic, such as a database indexing chapter.
2. Generate a Short Video Overview for the hardest concept and note any claim not found in your sources.
3. Open Reports, create an Interactive Learning Overview, and walk through it step by step.
4. Take a quiz using short answer and fill in the blank questions, then ask the notebook for feedback on where to focus.
5. Write down every fact in the outputs that you cannot trace to a source file, to judge how well grounding holds.

Small angles to try

• Add one source with a deliberate error and see whether the quiz repeats it.
• Compare quality on a small source set against a large one.
• Use the same notes with any other grounded notebook tool and compare which questions feel most useful.

Key ideas

CORS

A browser rule that blocks a web page from calling a server on another origin unless the server lists that origin as allowed.

KV cache

The stored attention keys and values for tokens already processed, which lets a model generate the next token without recomputing the whole prompt.

NVFP4

A 4 bit floating point format from NVIDIA that shrinks stored values so more context fits in GPU memory.

Steps

1. Read the Breaking changes section of the vLLM v0.32.0 release notes on GitHub first.
2. Open the vLLM quickstart docs and note the supported Python range and the default port 8000.
3. Skim the release notes again for the /v1/systemone entries and which models serve it.
4. Read the vLLM blog post on the Kimi K3 megakernel and note what was compared against stock vLLM.
5. Self check: if a browser app suddenly gets blocked after the upgrade, which server flag fixes it and why?

Steps

1. Create a fresh Python 3.11 virtual environment on your own machine and run pip install vllm, or pull the image with docker pull vllm/vllm-openai:v0.32.0.
2. Start a small model with vllm serve Qwen/Qwen2.5-1.5B-Instruct and wait until it listens on port 8000.
3. Send a request to http://localhost:8000/v1/completions with curl, setting a small max_tokens, and check the output length.
4. From a local web page served on another port, call the same endpoint with fetch and confirm the browser blocks it under the new CORS default.
5. Restart the server with --allowed-origins set to your local page origin and confirm the browser call now succeeds.

Small angles to try

• Send a long prompt with a large max_tokens and compare behavior with an older vLLM version.
• Run your existing client test suite against the new version before upgrading production.

Key ideas

MCP server

A program that exposes tools an AI agent can call through the Model Context Protocol.

Linked device

A secondary client, like WhatsApp Web, that the phone authorizes by scanning a QR code.

Durable Object

A Cloudflare Workers feature that gives one object its own storage, here a SQLite database.

Steps

1. Read the whatsapp-mcp README on GitHub first, including the section on terms and ban risk.
2. List the eight tools it exposes and note that none of them send messages.
3. Find where the README says read tools refresh when the stored copy is older than ten minutes.
4. Compare its pull on demand plus daily cron design with a server that stays connected all the time.
5. Self check: why does a read only, pull on demand design lower risk, and why is the risk still not zero?

Steps

1. Clone the repository with git clone https://github.com/finereli/whatsapp-mcp and run npm install.
2. Run npm run typecheck, then npm test; the README notes search is not covered because Node's built in SQLite lacks FTS5.
3. Read the tool definitions and confirm in code that no tool can send a message.
4. Read the sync logic and find the ten minute freshness rule and the daily 04:00 UTC cron.
5. Skip linking a phone. Linking would connect through an unofficial library that may conflict with WhatsApp's terms, so if you ever try it, use only your own account and accept that risk.

Small angles to try

• Sketch the same read only, store in your own account design for a data source you own, such as a local mail archive.
• List which tools you would remove or gate behind a password for a stricter version.

Key ideas

Claude Code plugin

A package of skills and instructions that Claude Code loads into a session to change how the agent works.

Hidden test suite

Tests the agents never see, used to score whether the finished build really meets the spec.

Seam check

A review of the interfaces where separately built tasks meet, where parallel work most often breaks.

Steps

1. Read the Superlite write up on contextfirst.blog first.
2. Open the Superlite README and read the benchmark table and its caveats.
3. Read the Superpowers README to see the workflow Superlite forked from.
4. Note which numbers are single runs and which are three run averages.
5. Self check: why is one 16.1 minute run weak evidence against a three run average with a 26.4 minute spread?

Steps

1. In a fresh Claude Code session without Superpowers enabled, run /plugin marketplace add iha2/superlite, then /plugin install superlite@superlite-dev.
2. Send Let's make a react todo list and confirm it starts brainstorming before writing code.
3. Write a short spec for a small tool plus a few tests you keep outside the project folder, and ask Superlite to build it.
4. Watch the .superlite/ folder in the project to follow tasks and recorded decisions.
5. Run your hidden tests on the result and note wall clock time and API cost.

Small angles to try

• Repeat the same spec three times and compare the spread of times.
• Run the same spec with Superpowers alone in a separate session and compare time, cost and test results.

Key ideas

Decision model

A model that returns a label, a score or a yes or no probability instead of free text.

Vendor reported benchmark

A score measured by the company selling the model, often on its own test set, so scores are hard to compare.

Labeled evaluation set

Your own examples with known correct answers, used to measure accuracy on your real task.

Steps

1. Read the MarkTechPost roundup first, focusing on its notes about self reported scores.
2. Read the TechCrunch piece for context on how quickly Jev was adopted on Vercel.
3. Pick three entries from the roundup and note price, license and whether weights are open.
4. Self check: why can a 0.93 score on one suite and 0.90 on another tell you little about your own classification task?

Steps

1. Collect 50 examples from your own data, such as support tickets, and label each with the correct category by hand.
2. Write a short prompt that asks for exactly one label from a fixed list.
3. Run the examples through two models from the roundup, or one roundup model and a general model you already use, saving each answer to a CSV.
4. With a short Python script, compute accuracy for each model and list the examples each got wrong.
5. Check the wrong answers for valid looking but incorrect labels, the failure the roundup warns about.

Small angles to try

• Add a confidence threshold and measure how accuracy changes when low scores go to a human.
• Compare cost per 1,000 examples using each provider's listed input price.

Key ideas

Descriptive versus causal

A descriptive trend shows what changed over time, while a causal estimate tries to show what caused the change.

Fixed effects

A statistical method that compares each person with themselves over time to remove stable differences between people.

Meta work

Work spent building, configuring and supervising agents instead of doing the task directly.

Steps

1. Read the Anthropic report on employment at Anthropic first.
2. Note how agent hours and human hours are defined, including the 40 hour week assumption and the cutoff for long waits.
3. Read the causal section comparing 192 adopters with 74 non adopters and the caveat that adoption was not random.
4. Read the HN discussion for outside reactions.
5. Self check: why does a 5x rise in commits per engineer not prove agents caused it?

Steps

1. Pick a repository you work on and choose two six month windows, one before and one after you began using coding agents.
2. Count your commits in each window with git log --author=<your name> --since=<date> --until=<date> --oneline, piping to wc -l.
3. Write down other changes in the same period, such as team size or project phase, that could explain any difference.
4. Estimate how much of a typical agent session you spend waiting versus reviewing, and compare it with the report's 80% idle figure.
5. Write two sentences separating what your numbers show from what they cannot prove.

Small angles to try

• Count reverted commits in each window to mirror the report's code quality check.
• Track for one week how many agent sessions wait on you at the same time.

Key ideas

Visual scratch space

The model treats the image as something it can revisit during reasoning, zooming into or measuring parts of it instead of describing it once.

Tool use on images

Actions like cropping, frame extraction or running code are tools the model calls mid answer, the same way text agents call search or a calculator.

Self reported benchmarks

Scores published by the model's maker on a chosen set of tests, useful as a hint but not a substitute for checking on your own data.

Steps

1. Read Elorian's announcement post "Introducing Glimpse" first and note which benchmarks Glimpse wins and which it loses.
2. Read the TechCrunch article for context on the team, funding and target markets such as robotics, labs and media.
3. Compare the pricing with the multimodal model your team uses today, per million input and output tokens.
4. Self check: why might a model that can crop and re read an image do better on dense scientific charts but worse on long video question answering?

Steps

1. Generate three charts from your own made up data with matplotlib: a dense line chart with small labels, a bar chart with 20 bars, and a scatter plot with a hidden outlier.
2. Write down the correct answers before testing, for example the exact value of bar 14 and the outlier's coordinates.
3. Sign in to the Elorian platform, open the chat interface and ask Glimpse each question with the matching chart attached.
4. Ask the same questions to the multimodal model you already use and record both answers side by side.
5. Score exact matches and note whether either model shows it zoomed in or measured anything before answering.

Small angles to try

• Use a photo of a page of your own handwritten notes and ask for a count of specific symbols.
• Shrink the chart resolution step by step to see where each model starts guessing.
• Record a short video on your phone and ask a question that depends on one brief frame.

Key ideas

Export control list

A government list of goods and technologies that need a licence before they can be transferred abroad.

Model weights

The trained numbers that make up a model; releasing them lets anyone run the model on their own hardware.

Parameter threshold

A rule that applies only to models above a certain size, here reported as 2 trillion parameters.

Steps

1. Read The Next Web article first for the scope, thresholds, exemptions and the comment deadline.
2. Skim the Techmeme cluster to see how different analysts read the draft, from Dean W. Ball to Kristy Loke.
3. Read the Rest of World piece for background on why Chinese open weight models matter to developers outside China.
4. Self check: under the draft as reported, which of these would be affected: an already released open model, a new open source code repository, or a future flagship model above 2 trillion parameters?

Steps

1. List every open weight model your team or projects depend on, with its publisher and the exact version you pinned.
2. For each one, open its model page on Hugging Face and record the parameter count and licence shown there.
3. Mark which models are above 2 trillion parameters and which come from publishers that would fall under the draft as reported.
4. Check that you keep a local or mirrored copy of every weight file you rely on, with its checksum and licence text saved next to it.
5. Write a one paragraph fallback note naming a second model you could switch to if a future version shipped only as an API.

Small angles to try

• Do the same inventory for embedding and reranking models, which are often forgotten.
• Compare how many of your dependencies would be affected by the reported 2 trillion threshold versus a hypothetical lower one.
• Track the draft until the 15 November comment deadline and note any changes reported by the opened sources.

Key ideas

Subagent

A separate agent the main session hands a task to, which works in its own context and reports back a result.

Fork subagent

A subagent that starts with a copy of the full conversation, system prompt, tools and model, so it needs no re explanation.

Headless mode

Running Claude Code non interactively with claude -p, typical for scripts and CI.

Steps

1. Read the v2.1.302 entry on the anthropics/claude-code releases page first.
2. Read the fork section of the Claude Code subagents documentation to see how forks differ from custom subagents defined in .claude/agents/.
3. Check the CLI reference for --permission-mode and --output-format so you know which defaults your scripts rely on.
4. Self check: if a CI script runs claude -p with no permission flag on a third party provider, which mode does it now start in, and how would you force auto mode?

Steps

1. Run claude update, then claude -v to confirm the version.
2. In a throwaway repository, run a headless task that invites delegation, for example claude -p --output-format stream-json --verbose "Summarize each file in src and then draft tests", and save the output.
3. Search the saved stream for Agent tool calls and note whether any use the fork subagent type.
4. Run the same prompt again as CLAUDE_CODE_FORK_SUBAGENT=0 claude -p --output-format stream-json --verbose "..." and compare the number of calls and total cost.
5. If you use a third party provider, run once without --permission-mode and once with --permission-mode auto to confirm which mode each starts in.

Small angles to try

• Add --forward-subagent-text to the stream json run to reconstruct each subagent's transcript.
• Define a small read only reviewer in .claude/agents/ and compare it with a fork on the same task.
• Set --max-budget-usd on both runs to see how forks affect spend under a cap.

Key ideas

Risk scoring model

A model that turns many input signals into a single number used to rank cases for attention.

Aggregation level

Whether a model scores individuals, teams or whole sites, which changes both its privacy impact and how decisions use it.

Proxy variable

An input that is not the target itself but strongly correlates with it, so a model can learn sensitive patterns indirectly.

Steps

1. Read the openDemocracy article first, noting what the documents say and what Amazon says in response.
2. Read the HN discussion to see how engineers reason about site level scoring and data sources.
3. List the inputs the documents attribute to Atlas and, for each, write what it could act as a proxy for.
4. Self check: why does a score that only rates sites still raise questions about the people who work there?

Steps

1. Generate a synthetic table of 500 fictional sites with columns such as survey satisfaction, overtime hours, region and a made up "forum activity" count, plus a random binary outcome.
2. Train a simple scikit-learn classifier on it and convert predicted probabilities into a 100 to 900 score and five tiers.
3. Use fairlearn's MetricFrame with selection_rate and sensitive_features set to region to see how often each region lands in the top tier.
4. Drop one input at a time, retrain, and record how the tier assignments change to find which features drive the ranking.
5. Write a short audit note on what the score can and cannot justify, and which inputs act as proxies.

Small angles to try

• Inject a deliberate correlation between region and the forum feature and see whether the audit catches it.
• Compare a site level model with one trained on synthetic individual records to see how aggregation changes the outputs.
• Add random noise to the survey column and measure how stable the tiers are.

Key ideas

Formal proof

A proof written in a language such as Lean that a computer checks step by step.

Readable versus verified

A proof can pass a checker and still be too opaque for a person to learn anything from it.

Disclosure standard

An agreed rule for stating how and where an AI tool contributed to a piece of work.

Steps

1. Read Maggi's guest post on Tao's blog in full first.
2. Make a two column list of the risks he names and the proposals he makes, and match each proposal to a risk.
3. Skim the HN discussion for responses from people who use AI in research or code.
4. Self check: why does Maggi argue that more capable proof generators increase rather than reduce the need for mathematicians?

Steps

1. Open the Lean playground and write a small statement you understand well, such as the sum of two even numbers being even.
2. Prove it yourself, or with help from documentation, until Lean accepts it.
3. Ask a chat model for a Lean proof of the same statement, paste it in and check whether Lean accepts it.
4. Compare the two proofs for readability and write a one paragraph explanation of the machine version in plain words.
5. Write a short disclosure note stating exactly which parts came from the model, as Maggi's proposal would require.

Small angles to try

• Repeat with an AI generated function in your own code base and write the same disclosure note for a pull request.
• Ask the model for a natural language version of its proof and check whether it faithfully matches the Lean steps.
• Try a slightly false statement and see how the model and Lean each respond.

Key ideas

Decision model

A model that reads an input and outputs probabilities over a fixed set of answers through a small readout head instead of writing free text.

Joint request

One forward pass that contains the shared state and several questions, each answered at its own marker token, so extra questions add little cost.

Calibration temperature

A single number used to rescale the model's scores so that a reported 0.8 confidence is right about 80 percent of the time; the card lists 1.382.

Steps

1. Read the pplx-decider-v1.2 model card on Hugging Face first, focusing on the joint format and the Decision Index table.
2. Compare the joint and one per request columns to see which task types gain or lose from joint requests.
3. Read the throughput and memory section to see what the fast path needs: CUDA, flash-linear-attention and torch 2.10 or newer.
4. Look at the flash v1.2 card to compare its 4.5B size against the 26B model.
5. Self check: why would a joint request be cheaper than sending each question separately, and on which kind of task did the card show it hurting?

Steps

1. Create a fresh virtual environment with Python 3.12 or newer on your own GPU machine.
2. Download the checkpoint with snapshot_download("perplexity-ai/pplx-decider-v1.2") from huggingface_hub.
3. Install the bundled source package from the downloaded checkpoint as the model card describes, using pip install "./source[fast]" if you want the fast path.
4. Load it with Decider.from_pretrained(checkpoint, device="cuda"), call decider.warmup(), then ask a few yes or no and multiple choice questions about your own text.
5. Send the same questions once as a joint request and once one per request, and record probabilities and latency for each.

Small angles to try

• Swap in pplx-decider-flash-v1.2 on a smaller GPU and compare answers with the 26B model.
• Use it to route your own support tickets into a handful of labels and measure accuracy against hand labels.
• Compare its confidence scores to a prompted chat model asked to answer with a single label.

Key ideas

Publicity rights

A legal claim that lets a person control commercial use of identifying traits such as a face or name, and possibly a voice.

Granular consent

Permission given separately for each purpose and stage, such as training versus publication, rather than one blanket yes.

Consent expiry

A rule that a permission stops being valid after a set period, here one year in principle, so it must be renewed.

Steps

1. Read the ITmedia AI+ article of October 16 first for the three pillars and the reactions.
2. Open the Nippairen guideline page and its PDF to see the rules on multilingual dubbing and deceased performers.
3. List which rules turn into system requirements: per stage consent, expiry dates, logs, deletion of data and models.
4. Read about the Tsuda ruling in the ITmedia piece to see why a court result and a ministry statement made the guideline timely.
5. Self check: if a performer withdraws consent, what three things must a voice service do under the guideline?

Steps

1. Design a small consent table in SQLite on your own machine with fields for performer, stage such as training or publication, purpose, granted date and expiry date.
2. Write a function that refuses a generation request unless a matching, unexpired consent row exists for that exact stage and purpose.
3. Add an append only usage log table and record every allowed or refused request.
4. Write a withdrawal routine that deletes the performer's stored sample files and prompt records, marks related model files as retired, and logs the action.
5. Run tests with fake records that cover expired consent, blanket consent attempts and withdrawal.

Small angles to try

• Add a rule that sharing data with a third party writes its own record, as the guideline asks.
• Generate a monthly report from the usage log that could support usage based payment.
• Compare this design with consent rules in a TTS service's published terms.

Key ideas

DiT

A diffusion transformer, an image generator that uses transformer blocks instead of a convolutional U-Net to remove noise step by step.

Layout conditioning

Giving the model boxes with captions so it places each object or piece of text in a chosen region of the image.

Classifier free guidance

A sampling trick that pushes the output toward the prompt by comparing guided and unguided predictions; CFG 3 sets its strength.

Steps

1. Read the MOSS-Image GitHub README first, especially the JSON layout example and its coordinate rules.
2. Read the MOSS-Image-SFT model card for requirements and sampler defaults.
3. Note how bbox coordinates are centered on the canvas and in yxyx order.
4. Compare the Base and SFT checkpoints to see which one the card suggests for direct use.
5. Self check: what must match between a JSON layout and the requested output size?

Steps

1. Clone the repo with git clone https://github.com/OpenMOSS/MOSS-Image.git, create a virtual environment and run pip install -r requirements.txt.
2. Load the model with diffusers DiffusionPipeline.from_pretrained("OpenMOSS-Team/MOSS-Image-SFT", custom_pipeline="pipeline_moss_image", trust_remote_code=True) and move it to CUDA.
3. Generate a 1024 by 1024 image from a plain prompt with num_inference_steps=50 and guidance_scale=3.0 and a fixed seed.
4. Write a JSON layout with two boxes, one holding a short line of text, pass it as the prompt with json.dumps(layout, ensure_ascii=False), and check where text lands.
5. Try the Gradio demo with pip install -e '.[ui]' and python gradio_app.py --model OpenMOSS-Team/MOSS-Image-Base.

Small angles to try

• Compare Base and SFT on the same seed and layout.
• Lower the step count and see where quality drops.
• Test text rendering in Japanese or Chinese inside a layout box.

Key ideas

Compile time LLM

Using a large model once to produce labels, rules and handlers, then never calling it again for each record.

Confidence gate

A threshold that makes the system abstain and pass a record on instead of guessing when its probability is too low.

Golden answers

A small set of records with known correct outputs used to check the compiled program before it goes live.

Steps

1. Read the Project-STROT README first, focusing on the difference between run and decide.
2. Read the benchmarks folder description to see how the compiled versus naive numbers were produced.
3. Read the HN thread for the main objection, that a generated script could do the same job.
4. Look up the Julia-1 decision model to understand what runs per record.
5. Self check: what happens to a record whose confidence falls below the gate?

Steps

1. Create a virtual environment and run pip install strot-ai[local] strot-ai[anthropic], and do not install the unrelated package named julia.
2. Prepare 50 of your own sample tickets or log lines with hand labels as golden answers.
3. Following the README quickstart, call strot.run() with your records, a plain language query and an action registry of two or three actions.
4. Run the compiled program over the records offline and record accuracy, abstentions and time per record.
5. Compare against simply prompting an LLM per record on the same set.

Small angles to try

• Use decide() with a boolean or score question instead of routing.
• Change the confidence threshold and plot accuracy against abstention rate.
• Ask an LLM to write a plain Python classifier once and compare it with STROT, testing the HN objection.

Key ideas

Mode collapse

When a generative model keeps producing a narrow set of similar outputs instead of covering the full range it was trained on.

Preference tuning

Training a model toward outputs that raters like, which can push results toward one popular average style.

Embedding diversity

A way to measure variety by turning each image into a vector and checking how spread out the vectors are.

Steps

1. Read the Yahoo! Realtime Search summary first for the post and the reply themes.
2. Read an introduction to mode collapse in generative models to connect the idea to training.
3. Think about which reply explanations, models, prompts or market strategy, could be tested and how.
4. Self check: how would you tell whether sameness comes from the model or from users writing similar prompts?

Steps

1. Write 10 varied prompts about the same subject, such as a portrait, without naming any style.
2. Generate 5 images per prompt with different seeds on your own machine and save them.
3. Embed every image with an open image embedding model and compute the average pairwise cosine distance.
4. Repeat with prompts that name distinct styles and compare the diversity scores.
5. Write down whether naming styles widened the spread and by how much.

Small angles to try

• Compare a base checkpoint and its tuned version on the same prompts and seeds.
• Raise and lower guidance strength to see its effect on diversity.
• Score diversity of your own sketches or photos as a human baseline.