What You Will Learn

By the end of this tutorial, you will know how to:

• create a simple qualitative codebook;
• turn the codebook into a structured AI decision;
• test responses one at a time;
• understand the AI’s classification and confidence;
• compare AI results with human coding;
• identify cases that need human review; and
• prepare the workflow for larger datasets.

1. The Problem We Want to Solve

Imagine that your organization conducted a survey and received 5,000 open-ended responses.

The question was:

You want to understand the main themes in these responses.

After reviewing a sample, your evaluation team decides to use these categories:

Now you need to assign a category to every response.

Doing this manually for 5,000 responses could be time-consuming.

This is where AI assistance can help.

2. First Build the Codebook — Before Opening TypeSafe

This is the most important beginner step.

A codebook is simply a set of categories and rules explaining what each category means.

Do not start by asking AI to “find themes.”

First decide what you want to measure.

Our example codebook

Why is the codebook important?

Imagine that you tell an AI:

That is too vague.

Instead, you tell it:

Now the task is much clearer.

3. Prepare a Small Test Dataset

Do not begin with your 5,000 real responses.

Start with 10–20 responses.

For this tutorial, use these examples:

The Human code column is your reference answer.

For a real evaluation, these classifications should come from your agreed coding process, ideally with appropriate quality assurance.

4. Open TypeSafe AI

TypeSafe currently provides a web interface at:

Jev is currently available through TypeSafe’s early-access offering. TypeSafe also provides an API for programmatic use.

5. Understand the TypeSafe Idea Before Clicking Anything

TypeSafe’s approach is different from opening ChatGPT and writing a long prompt.

The basic structure is:

For our qualitative coding example:

• Information: the survey response.
• Question: Which code best describes it?
• Possible answers: Access, Quality, Relevance, Gender, Inclusion, Sustainability, Other.
• Result: the selected category and associated probability/confidence information.

This is consistent with TypeSafe’s published description of Jev as producing typed decisions with probabilities and confidence, rather than ordinary generated text.

6. Create Your First Decision

When you reach the TypeSafe interface, look for the workflow/query area where you can define a question for Jev.

You want to create a Choice decision.

TypeSafe’s published workflow documentation identifies Choice as one of its core decision types: one option from a defined set, with a distribution across choices and confidence.

Your decision

Use a simple question such as:

Your choices

Add the seven allowed choices:

7. Give the Decision Some Context

The AI also needs to know what the categories mean.

Give it the codebook definitions.

For example:

This is important because the word “Access” could mean different things in different evaluations.

Your codebook gives the AI the meaning you want it to use.

8. Add the Survey Response

Now give the system the actual information it needs to classify.

For our first test, use:

Now run the decision.

The expected category, based on our human coding, is:

The purpose of the test is to see whether the AI reaches the same conclusion.

9. Look Carefully at the Result

Do not look only at the category.

Look at the additional information TypeSafe provides about the decision, particularly the probabilities/confidence.

Your result might conceptually look like:

Do not copy this as an actual TypeSafe result. It is an illustration of what you should examine in your own run.

TypeSafe states that Jev returns probabilities and confidence with its structured decisions.

10. Test All Five Responses

Now repeat the exercise.

11. Now Build a Simple Validation Table

Open Excel or Google Sheets.

Create these columns:

Your first rows might look like:

This spreadsheet becomes your validation record.

12. What If the AI Gets One Wrong?

Don’t immediately conclude that the AI is useless.

Investigate the disagreement.

For example:

Now we have an interesting methodological question.

Is this actually only “Gender”?

It could involve both Gender and Access.

Perhaps the problem is not the AI. Perhaps your codebook needs to allow multiple codes.

13. Decide: One Code or Multiple Codes?

This is an important methodological decision.

Option A — One code

Every response receives one primary category.

Example:

Option B — Multiple codes

A response can receive several categories.

Neither approach is automatically correct. Your evaluation design should determine which one makes sense.

14. Add Human Review

Now create a simple rule.

Do not choose a confidence threshold simply because it “sounds good.”

Use your validation sample to determine how confidence relates to actual accuracy.

15. Repeat the Test With More Data

Once your first five responses work, move to:

The larger your validation sample, the better you can understand where the workflow succeeds and where it fails.

16. Only Then Think About 5,000 Responses

Your final workflow might look like this:

17. What About Google Sheets?

This is where the tutorial can become more advanced.

Instead of manually copying responses into TypeSafe one by one, a later implementation can connect your spreadsheet to the TypeSafe API.

The idea would be:

This is the next level of the tutorial.

You don’t need to learn APIs to understand the concept. An API is simply a way for one software application to ask another software application to perform a task and return the result.

TypeSafe’s current customer agreement confirms that its services include both a hosted web interface and an API.

18. Privacy: Don’t Start With Real Sensitive Data

For your first experiment, use:

• fictional responses;
• publicly available data; or
• properly anonymized data.

Do not paste personally identifiable or confidential evaluation data into a new AI service until your organization has reviewed whether that use is permitted.

TypeSafe publishes privacy and data-processing documentation, but users still need to determine whether the service is appropriate for their specific project and data.

19. The Beginner’s Checklist

20. The Most Important Lesson

For qualitative coding, that means:

That is the foundation of a responsible AI-assisted qualitative coding workflow.

Official Resources

TypeSafe AI

TypeSafe Console

Introducing System One Models & Jev

TypeSafe Workflow Evals