Research software / October 2026
SCADS 2026
problem sets
Nine problems. Independent approaches.
From research plans to working software.
A chronological working record
How I worked through the problems
Follow one timeline across P01–P09. Each recorded outcome stays beside the prompt that directed it; connections link related project history.
Recorded session windows show elapsed calendar time, including gaps and overlapping sessions. They are not hours worked. The notes identify the timing evidence available for each step.
This log includes later development than some published source and paper snapshots. P06–P09 appear together as research proposals; their implementation had not begun.
The workflow could not load. Try reloading, or use the downloadable HTML and source JSON above.
The problem sets
05 in progress / 04 not startedOrganizational sensemaking
How can an agent understand an organization whose activities, dependencies, and claims are spread across many sources?
My solution. I built an evidence workspace that resolves an organization, follows its relationships, retrieves source material, and produces a brief with a traceable record of the investigation.
Conversational 311 analytics
How can an analyst ask questions about NYC 311 requests and receive inspectable summaries, geographic results, and reusable data?
My solution. I built a query service that turns structured analytical requests into filtered records, grouped statistics, geographic queries, period comparisons, and recoverable CSV exports.
Semantic discovery in technical manuals
How can a reader find equipment and technical specifications inside scanned manuals without exposing the original document collection?
My solution. I built VOPT, an offline pipeline that extracts candidate specifications, preserves their evidence, supports review, and releases a controlled metadata catalog for discovery.
Organizational knowledge graphs
How can communication records help an analyst explore organizational structure while keeping supporting evidence and uncertainty visible?
My solution. I built a local graph workspace with guided import, communication analysis, dated snapshots, source inspection, and recorded analyst reviews.
GraphRAG discovery
How can an analyst discover relationships in a changing corpus and distinguish new knowledge, real-world changes, corrections, and withdrawals?
My solution. I built a local evidence workspace that combines search, graph tracing, versioned sources, saved investigations, and comparisons between knowledge snapshots.
Agentic retrieval from selected resources
How could an agent enrich a presentation transcript with credible context about its speakers, topics, venue, and related talks from a predefined set of sources?
Projected solution. I propose an allowlisted research agent that plans its next lookup from the evidence it finds, resolves ambiguous identities, and produces a source-linked contextual dossier.
Implementation has not begun. The paper contains the proposed method, evaluation plan, and roadmap.
Efficient agentic RAG and evaluation
How could a lightweight agent decide when to retrieve, bridge languages, and produce grounded answers with an evaluation method that agrees with human judgments?
Projected solution. I propose a budget-aware retrieval controller paired with a separately evaluated judging framework for relevance, attribution, completeness, and cross-lingual faithfulness.
Implementation has not begun. The paper contains the proposed method, evaluation plan, and roadmap.
Task-specific AI capability benchmarks
How could a benchmark measure AI capabilities for specific research and software tasks without mistaking a public benchmark score for real-world readiness?
Projected solution. I propose a task-family benchmark with executable checks, evidence-based rubrics, separate human adjudication, and reserved evaluation items.
Implementation has not begun. The paper contains the proposed method, evaluation plan, and roadmap.
Core-to-edge model validation
How could a model retain useful quality and reliability as it moves from a development environment to resource-constrained edge hardware?
Projected solution. I propose a staged validation pipeline linking model optimization, controlled core tests, representative device measurements, release criteria, and lightweight monitoring.
Implementation has not begun. The paper contains the proposed method, evaluation plan, and roadmap.