Configure AI Application Analysis

AI Application Analysis enables administrators to configure criteria sets that the AI uses to evaluate and score applications. Once configured, analysis can run automatically at a trigger status or on demand, and results appear in a dedicated AI Analysis tab on each application record.

Who: UTA/Module Administrator, Global Administrator

When to Use AI Application Analysis

Use AI Application Analysis to:

• Reduce time spent on initial application screening by surfacing key insights automatically.
• Sort and prioritize applications in list view by AI-generated score, so reviewers can focus on the most promising candidates first.
• Identify flagged items and weak applications earlier in the review process.
• Gain deeper insight into application strengths and weaknesses with multi-dimensional scoring and narrative analysis.
• Configure analysis criteria to align with specific program goals and grantmaking priorities.
• Track scoring history over time to understand how applications evolve as criteria are refined or applications are updated and reanalyzed.

↑ Back to top

Open UTA/Module Configuration Settings

Once AI is enabled globally, administrators configure AI Analysis at the UTA/module level. Different criteria can be set for different grant types.

To access AI Analysis configuration:

1. Click the Menu icon in the upper navigation bar, and then select the applicable UTA/module.

2. Click the Configuration Settings gear for that UTA/module.

   

    

3. Click the General tab, locate the +AI section, and then click Application Analysis.

   

    

↑ Back to top

Create an AI Application Analysis Set

An Analysis Set is the top-level container that groups related criteria together and defines key settings that apply across all criteria in that set, including which fields the AI evaluates, which user roles can view results, and when analysis is triggered. Once AI Analysis is configured at the UTA/module level, administrators can create multiple criteria sets for AI Application Analysis. For example, one criteria set can be created for arts and culture grants and a separate criteria set for environmental grants.

To create an AI Application Analysis Set:

1. Click the New Application Analysis Set [+] button to create a new criteria set.

   

    

2. Add the applicable criteria set information:
   • Name - Provide a name for the criteria set.
   • Description - Provide a description to distinguish between sets (optional).
   • Trigger Status - Select the status at which AI analysis will automatically run (for example, on submission). The statuses available match those already configured in your system.
   • Supporting Files - Upload documents to provide the AI with broader domain knowledge for this set (optional). These files might include domain knowledge relevant to the grant type, such as a summary of what matters in arts and culture grantmaking. This context is used across all criteria in the set.
   • Allow View - Select which user roles will be able to see the AI Analysis tab and results. The user roles available match those already configured in your system.
   • Fields for AI Analysis - Select the fields the AI should analyze. Only the fields selected will be included in the AI Application Analysis. To keep certain fields out of the analysis (such as applicant name, contact details, or other personal information), leave them unselected.
3. Click Save.

↑ Back to top

Add Analysis Criteria

Once an Analysis Set exists, criteria are added to define the specific questions or dimensions the AI should evaluate. Each criterion is a discrete instruction that the AI applies across all fields selected in the set. For example, a criterion of "goal alignment" draws from every selected field to assess how well the applicant addresses it.

To add a criterion to a criteria set:

1. Click Criteria on the left-side menu to open the criteria set.

2. Click the New Criterion [+] button to add a new criterion.

   

    

3. Add the applicable criterion information. See Criterion Field Descriptions below.

   

    

4. Click Save.

Criteria can be edited or deleted after creation. To edit an existing criterion, click the edit pencil icon. Once analysis has been run, criteria can still be edited for future analyses. The scoring history will show how results change as criteria are updated.

Criterion Field Descriptions

Use the following field descriptions when entering information for a new criterion or editing an existing one.

• Name - Enter the name for the criterion. Keep the name brief.
• Type - Select the type of criterion.
  • Scoring (1-10) - The AI evaluates the application content and assigns a numeric score from 1 to 10 for this criterion.
  • Yes/No - The AI evaluates the application and returns either a score of 0 (No) or 10 (Yes) for this criterion.
  • Flag - The AI evaluates the application and flags items of note that may be worth a reviewer's attention, but are not given a numeric score.
• Weight - Select a weight (1x-5x) to indicate this criterion's importance in the total score. Use the edit function to assign weight after the criterion has been created.
• Evaluation Criterion - Enter a clear description for what the AI should assess. Additional qualifications for the 1-10 scale scoring can be added if needed.
• Supporting Files - Upload documents to provide the AI with specific knowledge for this criterion (optional). This might be a rubric or example relevant to one specific criterion only.

↑ Back to top

Sort Applications by Score in List View

Once AI analysis has been run, reviewers can add the AI score as a column in the UTA/module list view. This allows the list to be sorted by score, making it easy to prioritize which applications to review first.

To add the AI score as a column in list view:

1. Select the Default wrench icon from the Default drop-down menu on the UTA/module.

   

    

2. Click Columns and select Grant Fields from the drop-down menu.

   

    

3. Click the Add to new column [+] button for AI Application Analysis Score.

   

    

4. Click Save.

5. To sort applications by score, click the AI Analysis Score header. Click the header once to sort by descending score. Click the down arrow to sort scores in ascending order.

   

    

6. To hide the scores, click AI Analysis Score in the Edit Default List View screen and toggle Hide Column on.

   

    

↑ Back to top

Interactive Tutorial: Configure AI Application Analysis

This audio tutorial provides a click-by-click overview of how to configure AI Application Analysis.