Skip down to the research record

8 AI scorecard settings for uniform call reviews

Quota Forecast Editorial Team · Published · 10 min read

You can standardize call reviews with AI scorecards by configuring eight specific settings. According to Avoma, naming rules require a unique name so call participants can identify the correct scorecard for a particular meeting.

Key takeaways for how to set up ai scorecards for sales call reviews

  • Name each scorecard uniquely so call participants can identify the correct scorecard for a specific meeting, as Avoma notes.
  • Apply automatic review scorecards to calls that happened in the past 90 days and match the same filters, according to Gong.
  • Scale results to a value between 1 and 5, as described in Gong documentation.
  • Use filters to set the types of calls you want a scorecard to be available for, reported by Gong.
  • Allow AI to automatically fill scores and explanations using the conversation transcript for predefined or Pro level custom AI questions, as Aircall notes.

Standardize call reviews with AI scorecards

Uniform scoring depends on configuring eight distinct settings within your sales tool. The first setting involves unique names, which allow participants to select the right scorecard for a specific meeting. The second setting covers specific call filters that restrict which meetings receive a scorecard. The third setting defines AI question formats, telling the AI the output format: summary, bullet points, direct quotes, or analysis. The fourth setting establishes scoring scale limits, and Gong documents results scaled and rounded to a value between 1 and 5. The fifth setting configures automatic review windows, determining how long the system evaluates past calls. The sixth setting manages transcript access rules, defining what content the AI can read. The seventh setting covers custom questions, where you manually select an answer and optionally add comments. The eighth setting creates statistical aggregation views, summarizing scores across multiple calls.

Defining the scope and unique identifiers

To standardize how sales calls are evaluated, you must first define the unique identifiers that distinguish each scorecard. Avoma specifies that the name should be unique, as it will be used by the call participants to identify the scorecard to use for that particular meeting, as reported by Avoma. This naming convention ensures that the correct evaluation framework is selected during the call setup process. When configuring the scope of your reviews, you need to determine which specific interactions the scorecard applies to. Gong explains that you can click +Add filter to set the types of calls you want this scorecard to be available for, as reported by Gong. By applying these filters, you restrict the scorecard's availability to specific call categories, preventing it from appearing on irrelevant interactions.

The following table summarizes the eight settings required for uniform sales call reviews, with each row tied to the specific publisher that defines the rule.

SettingPublisherRule Detail
Unique namesAvomaName must be unique for participant identification
Specific call filtersGongAdd filters to set call types for availability
AI question formatsOutreachSpecify summary, bullets, quotes, or analysis
Scoring scale limitsGongScaled and rounded to a value between 1 and 5
Automatic review windowsGongApplies to calls from the past 90 days
Transcript access rulesOutreachAI only accesses call transcript, not post-call actions
Manual answer selectionAircallManually select answers and add comments for custom questions
Statistical aggregation viewsAircallGlobal scores for each category in Scorecard stats

When setting up your scorecards, avoid generic labels that might confuse participants during the call. Instead, use descriptive names that clearly indicate the purpose of the scorecard, such as "Discovery Call Review" or "Demo Evaluation." This practice aligns with the requirement for unique identification noted by Avoma. Similarly, when defining the scope, consider whether your scorecard should apply to all calls or only specific types, such as discovery calls or demos. Gong’s filter functionality allows you to make this distinction, ensuring that the scorecard is only available when relevant.

Configuring filters and call types

To restrict which interactions receive automated evaluation, you must define the specific call types included in the scorecard scope. According to Gong, you set the types of calls you want this scorecard to be available for by clicking the +Add filter option. This setting allows you to limit the scorecard's application to specific call categories, ensuring that the AI evaluation only applies to the interactions you intend to review. By configuring these filters, you prevent the scorecard from running on irrelevant call types, which keeps your review data focused on the specific sales conversations that matter to your team's performance metrics.

Read Weekly deal review filters across 4 sales systems.

When you add a filter, you are essentially creating a boundary for the scorecard's operational scope. Without these filters, the scorecard might apply to call types that do not fit your evaluation criteria, potentially introducing noise into your performance data.

Setting up AI questions and scoring scales

When you write questions for the AI to evaluate, specify the exact output format you expect. According to Outreach, you should tell the AI whether you want a summary, bullet points, direct quotes, or analysis. For example, if you want to identify customer issues, you can ask the AI to "Summarize the top 3 pain points mentioned by the prospect." Put this line in the prompt: "For each pain point, provide a 1-2 sentence description." This formatting rule helps ensure the AI generates structured text that fits into your scorecard without requiring extensive manual editing.

Gong states that the results are then scaled and rounded to a value between 1 and 5. This specific 1 to 5 range is documented for Gong's scorecards; do not assume other platforms use the same scaling without verifying their own help docs.

For certain question types, the AI handles the scoring automatically. Aircall notes that for scorecards containing predefined AI questions or Pro level custom AI questions, AI automatically fills scores and explanations using the conversation transcript. This means you do not need to manually input the score for these specific question types; the system generates the value based on the call content. If you are using custom questions that lack predefined AI logic or Pro level access, this automatic filling may not apply, so verify your plan level before relying on it.

Managing manual answers

Configure your scorecard to handle situations where automated analysis cannot capture the full context of a sales interaction.

Automatic review windows

When you enable automatic review, the system applies the scorecard to historical calls that fit your defined criteria. According to Gong, scorecards set to automatic review can be applied to calls that happened in the past 90 days that match the same filters. This specific 90-day window defines the maximum lookback period for automatic application. You should verify that your call volume and review schedule align with this timeframe. If you need to review calls older than 90 days, you must check whether your specific platform configuration supports manual backfilling or if the 90-day limit is a hard boundary for that particular feature.

Transcript limitations and post-call actions

It does not have visibility into actions taken by the sales representative after the call ends. As stated by Outreach, AI only has access to the call transcript, not what happened after. This limitation means you should avoid creating questions that ask about post-call behaviors, such as whether a rep updated the CRM. The source explicitly flags "Did the rep update CRM?" as an example of a question that fails because the AI lacks access to post-call data. Keep your question set strictly within the scope of the conversation to ensure the AI can generate accurate scores.

Manual answer selection

According to Aircall, for custom questions, manually select an answer and optionally add comments. The optional comment field provides a space to document why you chose that specific answer, which can be useful for training purposes or for auditing the review process later.

Reviewing results and statistical views

Once your scorecards are live, you need a way to see whether the scores are consistent across the team. According to Aircall, when clicking on a published scorecard in the Scorecard tab, users can access Scorecard stats. This is the entry point for reviewing how the AI has evaluated calls against your defined criteria.

The stats for each scorecard also show the global scores for each category, as reported by Aircall. This aggregation allows you to compare performance across different skill areas or sales stages. For example, if your scorecard includes categories for "Discovery" and "Objection Handling," the global scores let you see which area is lagging across the entire team. This view is particularly useful for identifying training gaps that are not visible when looking at single-call reviews.

See AI call scorecard checks: 5 rules before the forecast.

When you drill down into individual call results, the language of the AI’s response is a factor to consider. AI answers are filled in using the AI question language, according to Aircall. If your team operates in multiple languages, ensure your AI questions are set to the appropriate language so that the generated answers are understandable to the reviewers. This setting prevents confusion during the review process and ensures that the feedback is actionable for the sales rep.

For a more detailed look at specific interactions, you can access the call summary directly. Open the Briefs tab to read the summary for this call, as stated in Gong. It is a quick way to get context before you dive into the full transcript or the specific scorecard results.

Within that summary, you can view the summary, key points, and next steps, according to Gong. These elements help you verify that the AI’s scoring aligns with the actual conversation flow.

Illustrative example

One unique name lets participants identify the scorecard for that meeting. One filter selects discovery calls. The prompt asks for a summary of 3 pain points with a 1 to 2 sentence description of each. Results are scaled and rounded to a value between 1 and 5. Automatic review uses the past 90 days. Questions stay in the call transcript, not what happened after. One custom question gets one manual answer and one comment.

Next step

Draft a list of 5 specific, transcript-based questions for your next sales call review and map them to your current CRM fields.

FAQ: How to set up ai scorecards for sales call reviews

How long can automatic scorecard reviews apply to past calls?

Gong states that scorecards set to automatic review can be applied to calls that happened in the past 90 days that match the same filters Gong. This window applies specifically to calls meeting the configured criteria.

What is the maximum range for AI-scaled score values?

According to Gong, results are scaled and rounded to a value between 1 and 5 Gong. This defines the specific numerical range for the scoring output.

Can AI scorecards evaluate actions taken after the call ends?

Outreach clarifies that AI only has access to the call transcript, not what happened after the call ends Outreach. Therefore, the system cannot verify post-call actions like CRM updates.

How do I ensure the AI understands the specific format of my questions?

Tell the AI whether you want a summary, bullet points, direct quotes, or analysis, as Outreach describes. Aircall notes that AI answers are filled in using the AI question language Aircall.

Sources