ChartHeroAI

DocsDeveloper Docs

Evaluations

Evaluation definitions

What a definition measures, what a rubric expects of each role, and how to cache definitions.

When to use this

A definition says what an evaluation measures: its dimensions, score anchors and vocabularies. It is shared by every role. A rubric says what is expected of one role against that definition. Use definitions to label results, build lookup tables, and decide which results can be combined.

How evaluations, definitions and rubrics relateAn evaluation type has definition releases. A definition says what is measured and is shared by every role; it has one rubric per role and version, which says what is expected of that role, and more rubrics can be added later. Each evaluation names the definition_id and rubric_id that produced it, the conversation through encounter_id and document_id, the evaluated person through evaluated_actor_id, and its result.DESCRIBED BY GET /external/v1/evaluation-definitionsRETURNED BY GET /external/v1/evaluationsEVALUATION TYPEcare_coordination_conversationDEFINITION · VERSION 1What is measured7 dimensions, score anchors from −2 to +2and vocabularies. Shared by every role.RUBRIC · VERSION 1care_coordinatorWhat is expected of that roleRUBRICAnother role or version, added laterEVALUATIONPer conversation and person; more if reprocesseddefinition_idThe definition release: what was measuredrubric_idThe rubric release: what was expectedencounter_id · document_idThe conversation that was evaluatedevaluated_actor_idThe person who was evaluatedresultSummary, profile, points and findingsHow evaluations, definitions and rubrics relateAn evaluation type has definition releases. A definition says what is measured and is shared by every role; it has one rubric per role and version, which says what is expected of that role, and more rubrics can be added later. Each evaluation names the definition_id and rubric_id that produced it, the conversation through encounter_id and document_id, the evaluated person through evaluated_actor_id, and its result.DESCRIBED BY GET /external/v1/evaluation-definitionsEVALUATION TYPEcare_coordination_conversationDEFINITION · VERSION 1What is measured7 dimensions, score anchors from −2 to +2and vocabularies. Shared by every role.RUBRIC · VERSION 1care_coordinatorWhat is expected of that roleRUBRICAnother role or version, added laterRETURNED BY GET /external/v1/evaluationsEVALUATIONPer conversation and person; more if reprocesseddefinition_idThe definition release: what was measuredrubric_idThe rubric release: what was expectedencounter_id · document_idThe conversation that was evaluatedevaluated_actor_idThe person who was evaluatedresultSummary, profile, points and findings
An evaluation names the definition (what was measured) and the rubric (what was expected of the person's role) that produced it. One definition serves every role; each role gets its own rubric.

GET /external/v1/evaluation-definitions returns every definition your organization can see, active and retired, each with the rubrics your organization can see. It requires only evaluations:read, takes no query parameters, and is not paginated. Definitions are ordered by type, then version, and each definition's rubrics by role, then version. The route reads no patient data and is not audited as a PHI disclosure.

cURL
curl "https://api.charthero.ai/external/v1/evaluation-definitions" \
  -H "Authorization: Bearer <api_key>"

Any query parameter returns unsupported_filter. To receive the definition an evaluation references in the same response, request include=evaluation_definitions on an evaluation route instead.

Every dimension, score anchor and vocabulary value of ChartHero's standard definitions is listed in Standard evaluation definitions.

Example response

An abbreviated response with one definition and its rubric. The rubric guidance uses placeholder text; the API returns the real guidance.

JSON
{
  "items": [
    {
      "definition_id": "6f102fa4-ad28-4f20-b86d-b944ee32d7eb",
      "evaluation_type": "care_coordination_conversation",
      "definition_version": 1,
      "label": "Care coordination conversation",
      "description": "Measures how participants carry out care coordination in recorded conversations, relative to their roles and the opportunities the conversation presents.",
      "status": "active",
      "scope": "standard",
      "result_schema": "conversation_evaluation.v1",
      "methodology": {
        "score_anchors": [
          {"points": 0, "label": "Meets expectations", "description": "Competent and appropriate for the opportunity the call presented."}
        ],
        "dimensions": [
          {
            "key": "safety_risk_recognition_and_escalation",
            "code": "SR",
            "label": "Safety, risk recognition & escalation",
            "description": "Recognition and clarification of meaningful risk, with assessment, escalation, routing and interim guidance appropriate to the participant's role."
          }
        ],
        "profile_attributes": [
          {
            "key": "overall_complexity_level",
            "label": "Overall complexity",
            "description": "The highest known clinical, care-navigation or interaction complexity; unclear when none is known.",
            "values": [{"value": "moderate", "label": "Moderate"}]
          }
        ],
        "applicability_values": [
          {"value": "highly_applicable", "label": "Highly Applicable", "scored": true},
          {"value": "not_applicable", "label": "Not Applicable", "scored": false}
        ],
        "finding_values": {
          "level": [{"value": "positive_strong", "label": "Strong positive", "group": "positive"}],
          "category": [],
          "attribution": [{"value": "evaluated_participant_behavior", "label": "Evaluated participant behavior"}],
          "review_marker": [{"value": "safety", "label": "Safety"}],
          "practice_element": [],
          "focus_area": [],
          "issue_type": []
        }
      },
      "rubrics": [
        {
          "rubric_id": "81cb28c6-2c30-4473-acf4-1ff258030470",
          "rubric_version": 1,
          "evaluated_role": "care_coordinator",
          "label": "Care coordinator",
          "status": "active",
          "scope": "standard",
          "applies_to": "Recorded conversations by care coordinators that produced a Patient Outreach note.",
          "guidance": {
            "dimensions": [
              {
                "key": "safety_risk_recognition_and_escalation",
                "question": "Placeholder question for this role.",
                "look_for": ["Placeholder behavior."],
                "anchors": [{"points": 0, "description": "Placeholder description of meeting expectations for this role."}],
                "not_scored": "Placeholder description of when the dimension is left unscored."
              }
            ]
          }
        }
      ]
    }
  ]
}

Definitions

Field Type Meaning
definition_id string Permanent ID of this release.
evaluation_type string Type key, such as care_coordination_conversation.
definition_version integer Release number within the type.
label, description string Human-readable name and description.
status string active, or retired when new evaluations no longer use it.
scope string standard for ChartHero-maintained releases, or organization for your organization's own.
result_schema string Structure of its results, such as conversation_evaluation.v1.
methodology object What is measured, shaped by result_schema.
rubrics array The rubrics your organization can see for this definition.

For conversation_evaluation.v1, methodology has:

Field Shape Meaning
score_anchors {points, label, description}[] What each point value from −2 to +2 means.
dimensions {key, code, label, description}[] What is measured. Descriptions are role-neutral.
profile_attributes {key, label, description, values[]}[] The keys of result.profile and the values each can take.
applicability_values {value, label, scored}[] Values of dimensions[].applicability.
finding_values object Vocabularies for findings: level (with group: concern, opportunity, expected or positive), category, attribution, review_marker, practice_element, focus_area and issue_type.

Every vocabulary value carries value and label. Level values add group, applicability values add scored, and profile values add a description when one exists.

Rubrics

Field Type Meaning
rubric_id string Permanent ID of this release.
rubric_version integer Release number of this role's rubric within the type.
evaluated_role string The role it sets expectations for, such as care_coordinator.
label string Human-readable name.
status string active, or retired when new evaluations no longer use it.
scope string standard or organization.
applies_to string Which conversations get evaluated with this rubric.
guidance object What is expected per dimension, shaped by the definition's result_schema.

For conversation_evaluation.v1, guidance.dimensions has one entry per dimension in the definition:

Field Shape Meaning
key string The definition's dimension key.
question string The question the dimension answers for this role.
look_for string[] Behaviors the evaluator looks for.
anchors {points, description}[] What each point value looks like for this role. Their labels are the definition's score_anchors.
not_scored string When the dimension is left unscored.

Guidance summarizes the standard the evaluator applies to each role. These docs describe its shape only; read its content from the API.

Cache definitions by definition_id

IDs are permanent, and a release's meaning never changes. In place, a release can only gain vocabulary values or clearer wording. A change in meaning is a new release with a new ID, and the previous release stays, retired.

Rubrics are added to existing definitions: a new rubric version, a new role's rubric or your organization's own rubric appears under the same definition_id, and a replaced rubric becomes retired. So:

  1. Cache each definition by definition_id.
  2. Fetch the list again when an evaluation names a definition_id you haven't cached, or a rubric_id your cached definition doesn't list.
  3. Refresh the list when you reconcile evaluations. A release has no change marker. A refresh can add values or reword labels but never removes a value, so update lookup tables by definition_id, vocabulary and value. Rewording never changes a value's meaning, so overwrite labels; you don't need their history.
  4. Combine scores only within one definition_id.

Organization-specific entries

A definition or rubric with scope: organization belongs to your organization. Your organization can have its own rubric on a standard definition, whose results combine with standard results under the same definition_id, or its own evaluation type with its own dimensions.

Organization entries appear only in that organization's responses, never in these docs. Key your integration on IDs and read definitions from the API so it works for every configuration. Today every entry is standard.

Next steps

How evaluations work

Understand the evaluation resource and how to read scores.

ChartHeroAI

Developer documentation for ChartHero APIs, adapters, and integrations.

HIPAA-aligned safeguards

BAA available

ChartHero AI LLC