The Difference Between Audits and Insights
Before diving in, it's important to understand what makes Insights different from Audits:
Audits | Insights |
Measure documentation quality | Measure outcome progress |
"Is this note well-written?" | "Is this person making progress?" |
Score individual notes | Analyse the whole case journey |
Can have 100% audit scores with no progress | Reveals whether meaningful change is happening |
Good documentation ≠ Good outcomes
A case worker could write excellent notes every week while the person they're supporting remains stuck. Insights help you see whether your service is actually achieving the change you set out to create.
What is Theory of Change?
A Theory of Change is a description of how your service creates positive change. It answers:
What activities do we deliver?
Who do we work with?
What changes do we expect to see in people?
How do those changes lead to meaningful life outcomes?
The Six-Stage Framework
Insights analyse cases against a proven six-stage framework:
Stage | What It Measures | Example |
What we do | Are we delivering the planned activities? | Weekly support sessions, housing referrals, benefits advice |
Who with | Are we reaching and engaging the right people? | Person is attending, engaged, building relationship |
How they feel | Are there emotional/wellbeing improvements? | Less anxious, more hopeful, feeling supported |
What they learn and gain | Are they acquiring knowledge, skills, resources? | Understands benefits system, budgeting skills, knows options |
What they do differently | Is there evidence of behaviour change? | Attending appointments, managing finances, maintaining tenancy |
What difference does this make? | Are meaningful life outcomes being achieved? | Stable housing, improved mental health, sustained employment |
Each stage builds on the previous ones. Early stages (activities, engagement, feelings) are easier to achieve than later ones (behaviour change, life outcomes).
Setting Up Your Theory of Change
Where to Configure It
Go to Settings in the main navigation
Select Case Management
Find the Theory of Change section
Write your theory of change in the text box
Click Save
What to Write
Describe what success looks like for your service across all six stages. Be specific about:
What you do:
What activities and support do you provide?
How often do you meet with people?
What resources do you offer?
Who you work with:
Who is your target group?
What does engagement look like?
How you expect people to feel:
What emotional changes indicate progress?
What wellbeing improvements do you aim for?
What you expect people to learn and gain:
What knowledge should they acquire?
What skills should they develop?
What resources should they access?
What you expect people to do differently:
What behaviour changes indicate progress?
What actions should they be taking?
What difference this makes:
What are the ultimate outcomes you're working towards?
What does success look like in people's lives?
Example Theory of Change
Our service provides weekly one-to-one support sessions, housing referrals, benefits advice, and emotional support.
We work with adults experiencing homelessness or at risk of homelessness, often with complex needs including mental health challenges and substance use.
We expect people to feel: less anxious, more hopeful, supported and not alone, increased confidence in their ability to make changes.
We expect people to learn: how to navigate the benefits system, budgeting and money management skills, understanding of housing options and tenancy responsibilities, self-advocacy skills.
We expect people to do differently: attend appointments with services, engage with treatment or support programmes, manage their finances more effectively, maintain tenancy responsibilities, reduce harmful behaviours.
Ultimately we expect to see: stable and sustained housing, improved mental health and wellbeing, reduced crisis incidents, engagement in employment, training or meaningful activity.
AI Feedback on Your Theory of Change
After writing your Theory of Change, click Analyse ToC to get AI feedback:
Is each stage covered adequately?
Are there gaps or vague areas?
Suggestions for making it more specific
The AI will show a coverage assessment for each stage:
Good — Stage is well described
Needs detail — Some information but could be clearer
Missing — This stage isn't addressed
Use this feedback to strengthen your Theory of Change before running case analysis.
Understanding the Insights View
The Heatmap
The main Insights view shows a heatmap with:
Rows — Individual cases
Columns — The six outcome stages
Colour — Progress level (red = low, amber = medium, green = high)
Opacity — Confidence level (faded = limited evidence, solid = strong evidence)
This gives you an instant visual of:
Where cases are progressing well (green)
Where cases are stuck (red/amber)
Where you have limited evidence (faded colours)
Progress Levels
For each stage, the AI assesses progress as:
Low — Little or no evidence of progress at this stage
Medium — Some progress but not yet fully achieved
High — Strong evidence that this stage has been achieved
Confidence Levels
Confidence reflects the quality and quantity of evidence:
Low — Few interactions to analyse, or limited relevant information
Medium — Reasonable evidence but some uncertainty
High — Plenty of interactions with clear, relevant information
Why confidence matters: A "Low progress" assessment with "Low confidence" might mean there's actually progress but it's not being recorded. A "Low progress" assessment with "High confidence" is more concerning — there's plenty of evidence showing the person is stuck.
Drilling Down
Click on any cell in the heatmap to see:
The AI's summary for that stage
Key evidence from case notes
What would indicate progress at this stage
Click on a case name to see its full insight analysis including:
Progress at all six stages
Overall summary
Key strengths
Areas for attention
Generating Insights
When Insights Are Generated
Insights are not automatically generated. You need to trigger analysis:
Go to Cases > Insights tab
Click Generate Insights
The AI analyses all open cases against your Theory of Change
Results appear in the heatmap once processing completes
What Gets Analysed
For each case, the AI reviews:
All case notes
Survey and assessment responses
Event attendance records
The case timeline and history
It looks at the most recent three months of interactions by default.
Caching and Freshness
To save processing time, insights are cached:
If a case has no new interactions since the last analysis, the cached result is used
If new interactions have been added, the case is re-analysed
You can force re-analysis by clicking Regenerate on individual cases
A "stale" indicator shows when a case has new interactions that haven't been analysed yet.
Aggregate Views
Team Comparison
The Insights tab can show analysis by case worker:
See each worker's caseload performance
Compare progress patterns across the team
Identify if certain workers need support
Funnel View
Shows the percentage of cases at Medium or High progress for each stage:
Visualises where cases "drop off" in the outcome journey
Helps identify systemic gaps in your service
Shows where to focus improvement efforts
Narrative Summary
The AI generates an overall narrative about your caseload:
What's going well across your cases
Common challenges and patterns
Areas where multiple cases are stuck
Using Insights Effectively
Regular Analysis
Run insights analysis regularly (e.g., monthly) to:
Track progress trends over time
Catch cases that are stuck before they become crises
Demonstrate outcomes to funders and stakeholders
Supervision Discussions
Use case insights in supervisions to:
Move beyond "what happened this week" to "is this person progressing?"
Focus support where it's most needed
Celebrate cases that are achieving meaningful change
Service Improvement
Aggregate insights reveal patterns:
If many cases are stuck at the same stage, consider whether your service model needs adjustment
If confidence is consistently low, consider whether note-taking practices need improvement
If certain case workers have different patterns, explore what's working well
Realistic Expectations
Remember that later stages (behaviour change, life outcomes) take time:
Early-stage progress (engagement, wellbeing) is expected within weeks
Later-stage progress (behaviour, outcomes) may take months
Not every case will reach "High" on every stage — that's normal
Theory of Change vs Audit Criteria
These serve different purposes and should be configured separately:
Theory of Change | Audit Criteria |
Describes what success looks like | Describes what good documentation looks like |
Used for Insights analysis | Used for Audit scoring |
Focuses on outcomes | Focuses on note quality |
"Are people getting better?" | "Are notes well-written?" |
Both are important:
Good notes (measured by audits) provide the evidence
Progress towards outcomes (measured by insights) shows impact
If audit scores are high but insights show low progress, it might mean:
Notes are well-written but the service isn't achieving change
Or the Theory of Change needs to be more specific
Or the service model needs reviewing
If insights show good progress but audit scores are low, it might mean:
Great work is happening but not being captured well
Note-taking training is needed
Or audit criteria are too demanding
Frequently Asked Questions
Do insights update automatically? No. You need to click "Generate Insights" to run analysis. Results are cached until new interactions are added to a case.
How long does analysis take? A few seconds per case. For large caseloads, bulk analysis may take several minutes.
What if I haven't written a Theory of Change? The AI will use a generic framework, but results will be much more useful if you've described what success looks like for your specific service.
Can I see insights for closed cases? Yes, insights are preserved when cases are closed, showing the final state of progress.
Why is confidence low on some cases? Low confidence usually means:
Few interactions to analyse
Notes don't contain much relevant information
The case is new and there isn't much history yet
Can insights replace manual assessment? Insights provide a useful overview, but shouldn't replace professional judgement. Use them to prompt conversations and identify patterns, not as definitive verdicts.
How often should I run analysis? Monthly is a good rhythm for most services. More frequent analysis (weekly) might make sense for short-term interventions.
What if I disagree with an insight assessment? The AI bases assessments on what's recorded. If you disagree, consider:
Is relevant progress being captured in notes?
Does your Theory of Change need more detail?
Would different language in notes help the AI understand?
Summary
Theory of Change and Insights help you answer the fundamental question: Is our service actually making a difference?
By defining what success looks like and regularly analysing case progress against that framework, you can:
Identify cases that need more attention
Demonstrate impact to funders and stakeholders
Continuously improve your service based on what's working
Ensure you're not just busy, but genuinely effective