Overview

This recipe describes building an assessment app in Talent App Store.

An assessment is some interaction relating to a job application at some point during the recruitment process, for example:

• undertaking a skills assessment
• carrying out reference checks
• cultural/fit analysis
• carrying out medical checks or drug tests
• capturing video interview
• gamification interaction
• probity check
• HR-XML assessment

To complete an assessment, any of these parties might get involved along the way:

• the candidate themselves, e.g. an online questionnaire where your app captures data to calculate cultural fit
• a user, e.g. a recruiter specifying which specific pre-employment checks they want
• you, the developer, e.g. you are a fact checking firm that verifies that the details in a candidate's resume are accurate
• some back end code, e.g. computer program that scours social networks and web sites, scraping the online footprints of the candidate

See also:

• A demo assessment app, written in Java/Spring.
• A demo dynamic image generator for assessment apps, written in node.js.

Commercials

As the vendors of the assessmenthub app, Aotal charges you, the developer, for every API call to POST /assessments/byID/{id}/appDetails, with a completed assessment.

Fees will be based on either an agreed % of your total subscription fees to customers, or an agreed % of your fees per completed assessment, and will be designed to fit in with your existing fee model. The minimum fee per completed assessment will be $US1.00. There are no charges for partial/incomplete assessments.

As for any app in Talent App Store, your app also pays for core API calls, as described in "Commercials and pricing". These costs are low.

Set up

Lets look at assessments in action by installing some apps.

Note: when installing apps, the apps that have links are sandbox apps that you install via their install token - the other apps can be found publicly listed (find them with "Browse apps").

Install apps and prepare test data

First, follow these instructions to install Developer ATS and create a job application. Then:

1. Return to Developer ATS and you should see the incoming job application (in the Incoming bucket)
2. There are no assessments showing in the assessment strip, so lets add one.
3. Install the Picket assessment app. The picket assessment measures a candidate's ability to deliver customer service by asking them to choose from several images representing good service.
4. Install the Assessment app (user) assessment app.
5. Click refresh on the job application. Now, against the application, you should see the assessment strip display 3 boxes, one for each assessment type (the picket assessment has one type, the other assessment app has 2).

Assessment with email to candidate

1. Click on the picket assessment to start it. Note the picket image now has a dashed border, indicating the assessment is underway.
2. You should have received an email with a link to start the assessment. Complete the assessment.
3. Return to the ATS and refresh the page. The assessment should now be shown with a solid border, showing it is complete.

Assessment that auto-completes, with feedback to user

1. Click on the credit card icon to start that assessment.
2. The assessment completes instantly.
3. Clicking on the assessment shows a panel with the candidate's credit score.

Assessment that needs user input before completing

1. Click on the police hat icon
2. The assessment auto-starts, but does not complete (border remains dashed) - a triangle icon indicates the user needs to do something
3. Clicking on the assessment shows a panel for the user's input (i.e. which specific checks are required)
4. After providing the data, the assessment completes, and results are now available

The roles of each app

+----------------------
|
|   +-----+         +----------------+
|   | ats |<------->| assessment app |-+
|   +-----+         +----------------+ |-+
|     /\              +----------------+ |
|      |                +----------------+
|      |                  /\
|      |                   |
|     \/                  \/
|    +------------------------+
|    |    assessment hub      |
|    +------------------------+
|
+--- (tenant) ---------

Your assessment app. Your app does the hard word of actually conducting the assessment. For example a basic numeracy testing assessment might guide the candidate through a series of web pages, each one asking them an arithmetic question, and then at the end calculate an overall score for them and update the assessment's status in the hub to complete.

The assessment hub

The assessment hub app provides centralized assessment management and is separate from any single assessment app. The assessment hub app:

• stores the tenant's installed assessment apps - for example, the "Microsoft Office proficiency assessment app"
• stores a tenant-specific list of assessment types within each assessment app - for example "MS Word proficiency - entry level", "MS Excel proficiency - guru"
• provides, via API, strips that the ATS can embed in applicant lists to start/view assessments
• informs, via API, your assessment app when a new assessment is started
• skips assessment creation when an existing assessment for the candidate is available (reduces per-assessment costs and candidate inconvenience)
• provides APIs for your assessment app to fetch and update the state of the assessment object
• stores summary current state of all assessments - status, pass/fail and the status image
• securely mediates assessment app access to specific fields of the candidate/application, and prevents assessment apps from probing candidates they are not authorized to access
• provides a UI for the tenant to map assessment types to specific workflows (e.g., that the "sales skills 101" assessment type is only available for call center jobs), and to mark assessment types as being auto-started for new job applications

The applicant tracking system (ATS) app. The ATS:

• holds job applications and displays them as normal
• for each application, fetches assessment strip from assessment hub via API and displays
• via the assessment strip, the user can start and view assessments directly within the ATS pages.

API flow to/from your app

Implementing the APIs

Produce POST /tenants

The first API call incoming to your app is to tell you that a tenant has installed it.

Typically you might handle this API call by inserting into your customer table, sending an email to customer support team, etc. You might show the "setup required" icon and a settings page to facilitate this.

See installing and controlling setup and launch pages for more.

Produce GET /assessmentTypes/forApp

Example

GET /assessmentTypes/forApp

Response:

Detail

Returning your assessment types

Immediately after your app is installed, the assessment hub, a separate app, consumes GET /assessmentTypes/forApp on your app to ask for your assessment types.

In this example, your app is declaring that it supports a single assessment type, "MS Word proficiency".

Details that can be passed about each assessment type

See the assessment type schema for a detailed description of every property you can pass for your assessment types.

In this example:

• Assessment results can be reused within 365 days. So if a candidate completes the assessment for one job, and then applies for another within this time, they won't be asked to complete the assessment again.
• The assessment results are sortable within the user's UI. So your app must be prepared to respond to POST /assessments/sorts.

If your app needs setup

If your app is completely "on demand" then it will be ready to return its assessment types immediately after the app is installed.

Otherwise, if your app requires setup (e.g. you need to set up an account for the customer) before it can meaningfully offer any assessment types, then you should return an empty set of assessment types.

Then, later on when your app's setup is complete, tell the assessment hub that you have new assessment types by consuming POST /assessmentTypes/forApp/deltaPings/1. In response the hub will consume GET /assessmentTypes/forApp on your app.

How many assessment types?

Apps can use different strategies for their assessment types:

1. highly customised assessment types, e.g. each tenant has their own unique set of assessment types, tailored to their needs (e.g., "Acme 2018 finance graduate assessment").
2. a single, universal assessment type that is the same for all tenants installing the app (e.g. "MS Word proficiency").
3. some combination of the above.

Making assessments reusable

This example shows a reusable assessment type. When your assessment type is marked as reusable, an assessment result on one job application is reused when the same candidate makes another job application within "daysExpired" - i.e the candidate only completes the assessment once.

For example, if the candidate has completed a medical check for one job, and then applies for another a few days later, we don't want them to be asked to complete the same check again. In this example, the assessment is reusable, and if the candidate completed the previous assessment within one year, then that previous result will be reused.

Most assessment types should be reusable. Before deciding that your assessment type should not be reusable, ask yourself "if a candidate completes my assessment and then a few minutes later they apply for another, identical job, does it make sense for them to complete the assessment again?".

Produce POST /assessments/byID/{id}/tenantDeltaPings

Example

POST /assessments/byID/{id}/tenantDeltaPings

Request:

Detail

At some point, an assessment for one of your assessment types will actually be started, e.g. when a recruiter selects it from a drop-down menu, or when it is automatically started as a result of a candidate dropping into a specific bucket.

This results in an assessment being created in the assessment hub.

The assessment hub then notifies your app by consuming your POST /assessments/byID/{id}/tenantDeltaPings endpoint.

Learning about new assessments

Note that although the API doc. specifies a request body, currently no body is sent. You should assume that if the assessment does not already exist in your own database, then this is a new assessment, otherwise it is an update to an existing assessment.

You might do your own system's housekeeping for the new assessment, e.g. creating rows in your own tables, sending messages, etc.

Updates or deletes to existing assessments (future)

In the future, your app may be informed via POST /assessments/byID/{id}/tenantDeltaPings that an assessment has been deleted (e.g. in response to a privacy response, or to conform to data retention laws). In response, your app should clean up data held its own database relating to the assessment.

Also in the future, your app may be informed that an assessment has been updated (e.g. in response to the tenant putting it on hold), in which case your app should stop whatever it is doing with the candidate.

Consume GET /assessments/byID/{id}

Example

GET /assessments/byID/{id}

Response:

Detail

Once your app learns that the customer has started an assessment, typically, you'll fetch the details of the assessment by consuming GET /assessments/byID/{id}.

The data you get back is mainly about the assessment itself (e.g. who raised it, if anyone), rather than the candidate. An important part of the response is the view key. You use the view key in the next step to fetch data about the candidate (enail, phone number, name etc.).

Consume GET /applications/views/byKey/{key}

Example

GET /applications/views/byKey/{key}

Response:

Detail

Many assessment apps will need specific details about the candidate, such as their CV, address, name, etc., and maybe more secure information such as salary, or customer-specific information such as "Suitability for warehouse roles". Your app gets these details from an application view.

You can learn about views here.

The assessment details that you fetched previously included a view key - pass that in to your call to GET /applications/views/byKey/{key} to fetch the correct view for this assessment.

Since the customer has complete control over the actual data you see in the view when your assessment is started, you need to be very clear in your documentation what data you require. For example, you might say in your app's description that you need job ID, candidate first name, last name and email, and recruiter email.

Missing candidate information

To do its work, your assessment may need information sourced from the candidate.

For some information, such as first name and last name, you can reasonably expect to find that inside the view. Its unlikely that the customer would run a recruitment process that didn't involve having the candidate's name.

For other information, such as the candidate's date of birth, the information might or might not have been captured in the recruitment process. You'll probably need to talk to the customer to establish whether the information is available in the view or not

For this kind of information, you can choose to either:

1. Put the assessment into the Error status, explaining to the customer what information is missing. Then the customer must somehow capture the information and then restart the assessment. Once they've done that, you'll see another incoming call to POST /assessments/byID/{id}/tenantDeltaPings. While this is easiest for you, it may be hard for the customer, as they may have to alter the setup of their site, change their existing processes etc., to capture the data.
2. Capture the information yourself directly from the candidate, e.g. by sending them a link to an online form where they fill in the information. This is easier for the customer, and makes your app much more "on demand". However the information you capture remains private to your app, and can't be reused elsewhere in the recruitment process.

Consume PATCH /assessments/byID/{id}/appDetails

Example

PATCH /assessments/byID/{id}/appDetails

Request:

In this example, your app updates the assessment's status to indicate it is underway, and sets the candidate interaction link to point to an online assessment.

Detail

Having been alerted to a new assessment, and having fetched all details for the assessment, your product can do its thing for the assessment.

At each stage of processing, your app should push the assessment's state up into the assessment hub by consuming POST /assessments/byID/{}/appDetails.

The assessment's state includes:

• Status
• Status image
• Candidate interaction uri
• User interaction uri
• Pass/fail flag (if applicable)

Status

An assessment can be in any of these statuses. There are 2 statues that can be set by the hub, and 4 statuses that can be set by your app.

Some statues are final, i.e. once the assessment is in this state, it will never leave it again.

Below are some typical scenarios:

Once the candidate has completed the online process hosted at your app, your app pushes the results, with the updated status.

In this example, the assessment is now complete. It could also have been that the app was just posting interim results.

Image

The assessment's image appears in the assessment strip, which in turn is usually embedded in the ATS alongside the other applicant details (name, email etc.). Setting the image is a good way for you to help the user understand the results from your assessment at a glance, without needing to drill in (unless they choose to). This is useful in the tightly packed screens in a typical ATS, which show many applicants and many assessments on a single page.

You can simply serve up a static image, or generate your images dynamically, e.,g. passing in the candidate's score as a URL parameter. For an example of this, see this demo dynamic image generator for assessment apps, written in node.js.

Users will appreciate an image which is a great visual representation of your assessment:

• A video interviewing app could render a video thumbnail.
• A psych assessment could render text with a percentage score, over a solid background in their corporate theme.
• A multi-stage probity test could render a series of checkboxes, with more boxes ticked as the assessment proceeds
• A culture fit test might show a colored graded heat gauge, with the needle pointing to the candidate's score
• A document check app could show a stamp, signifying complete

The status image should be suitable for displaying at an effective resolution of 52x32. Ideally your image should be 2x this to cater for high density displays (Retina Macbooks etc).

Candidate interaction uri

Previously the assessment hub could send an email to the candidate on behalf of your app, which could include the candidate interaction url.

This functionality has been deprecated, but you might still want to set the candidate interaction url (to e.g. the url for your quiz), as in the future there may be a UI that allows the candidate to reorient themselves from e.,g. within the ATS, and find their way back to you assessment.

You can optionally protect your web pages with TAS SSO, using principal type of candidate. This protects against the link falling into the hands of someone other than the candidate, and means the candidate enjoys a seamless experience as they navigate into your app.

If you do use SSO, you should probably perform a sanity check that the candidate who logs in matches the one the assessment is for.

userInteractionMessage

You should set the user interaction uri and/or message when you want to convey or display something to the user (i.e. someone looking at a list of applicants, and seeing your assessment there).

The message is an easy and quick way to display an error message, a link to your ractual results (e.g. in pdf format), or just a short message about the assessment.

The message is treated as markdown format, and you can include links (these will open in a new tab/window).

userInteractionUri

The uri allows you to provide a complete web page. This might show:

• The results of your assessment (status == Complete)
• The reason why your assessment couldn't be completed (status == Error)
• Something the user has to do before the assessment can proceed (status == In progress)

When userInteractionUrl is set, your app's web page is rendered within an iframe in the popup over the assessment strip (i.e. within the ATS pages).

The iframe has a minimum width of 380px by default, and a maximum width of 960px. If no height is supplied the browser will default to 150px.

You can update the dimensions of the iframe displayed in the popup, to make it best fit your content, with something like this:

var messageJson = {
    action: 'updateIframe',
    css: {
        height: document.body.clientHeight + 'px',
        width: '960px'
    }
};
window.parent.postMessage(messageJson, '*');

This technique can only be used to adjust width/height values. Any other CSS passed will be ignored (e.g. border).

Common problems getting your iframed content to appear

The following problems may cause a blank iframe inside the assessment popup.

• Make sure your content is served as https, not http. Otherwise in some browsers you may see errors in the browser such as "Mixed Content: The page at 'blah' wa loaded over HTTPS, but requested an insecure resource 'blah'. This request has been blocked; the content must be served over HTTPS."
• Make sure your page does not include the response header X-Frame-Options (e.g. something like X-Frame-Options:SAMEORIGIN). This is a security measure designed to stop your page from being framed - but in this case, that's exactly what you want.

Consume GET /candidates/me/homePage

Example

GET /candidates/me/homePage

Response:

Finally, after your app has done its thing and completed the assessment, it should redirect the candidate's browser to the candidate "home page", served by some other app. Their home page might direct them to the next assessment they need to complete, or otherwise keep them updated.

Consume POST /assessmentTypes/forApp/deltaPings/1

POST /assessmentTypes/forApp/deltaPings/1

From time to time, your app may introduce new assessment types, e.g.:

• After install, you've now completed the onboarding process for the new customer, so you can now make their assessment types available
• the tenant asks you for a new custom assessment type, and you have just delivered it

When this happens, you need to consume POST /assessmentTypes/forApp/deltaPings/1.

This API is produced by the assessment hub. The assessment hub will then consume GET /asessmentTypes/forApp on your app, to load your newly updated set of assessment types.

Produce POST /assessments/sorts

POST /assessments/sorts

Note: sorting is not yet implemented within the assessment hub, however you can implement this API now to future-proof your app.

When your app's assessment results are sortable (i.e. you've said so in your assessment type), then your app may receive incoming API requests whenever the user clicks in their UI to sort their job applicants.

For example, your assessment gives applicants a score of 0 - 100 based on a quiz result. The user clicks to sort applicants by your results. An API call is made to the POST /assessments/sorts endpoint in your app.

In the request, your app receives an unsorted array of assessment IDs, all of which will be for the same assessment type, e.g.:

POST /assessments/sorts

Request:

Your app can now sort the assessments, and return the sorted set, e.g.:

Response:

In this example response from your API:

• one of the assessments was not completed (has no result), so it is returned that in the unsortable array.
• two of the assessments have exactly the same sort value, so they are returned within a single sort result
• the other assessments have a unique sort value, so they each have their own sort result