EconJobMarket®

Step 1: Create rating scheme

1. Click on Rating schemes in horizontal menu.
2. In the simplest setup, you create one rating scheme, and each evaluator records their own rating of applicants, which only they can change. They can choose when to make their evaluations visible to all Reviewers (including all Managers).
3. Click the button Create rating scheme.
4. Give your scheme a name — shorter is better. Maybe "Quality"? Or "Rating"?
5. Choose whom you want to be able to record and see the ratings: probably in both cases, managers and reviewers.
6. Choose the number of levels for your rating scheme — 5 or 10 is probably good.
7. Choose the type of scheme — in the simple case that we are describing, choose "individual".
8. Do you want to allow reviewers to add comments? (Probably you do.)
9. Is this scheme going to be your default rating scheme, which you want to be attached to all future postings? (You can always change this setting later, and detach the scheme from particular postings.)
10. Click Continue, which takes you to a page where you can choose a label for each rating level. If you want to display these labels to Reviewers rather than the numerical ratings, entering labels is mandatory. Otherwise doing so is optional.
11. Click Save rating scheme. You will see it at the bottom of the Rating schemes page, and you can edit it if you wish.

Step 2: Attach rating scheme to position

1. Go to your Postings page.
2. Click the Rating schemes button for the posting for which you want to evaluate applications.
3. For the rating scheme you just created, click attach.

Step 3: Rate applicants

1. Each Reviewer (including all Managers) can now rate applications using the scheme you have created. To test it yourself, click on on the link Applications under the title of the page ("Attach rating scheme ..."). (Alternatively you can go back to your Postings page and click on the link to applications for the relevant posting. The link says n applications, where n is a positive integer.)
2. Click on the link Ratings below the title of the page. (You can also rate applicants by clicking the links on the current page, but the Ratings page has features designed specifically to display ratings.)
3. Click on the name of an applicant.
4. Near the bottom of the resulting window is a panel titled Evaluation.
   • Choose a rating and optionally add a comment (if you enabled that field when you created your rating scheme).
   • Click Submit.
   • If you are ready to make your ratings visible to all Reviewers, click the link Make ratings visible to all reviewers in your organization.
   • Close the window and go to another applicant. Subsequently when you have evaluated an applicant you can click the Submit and close window button to record your rating and close the window in one fell swoop.
5. The Ratings page for a position allows you to order applicants by various criteria. Its appearance will depend on how many Reviewers you have. (If there are too many to display the ratings of each one in a column, the ratings for each applicant are displayed as a list.)

1. You can specify as many schemes as you like, and attach more than one to any given posting.
2. Here are two possible setups involving two schemes. In both cases you create one individual rating scheme, as in the "Simple setup" described above, and also one common rating scheme. (To create such a scheme, follow the steps in the "Simple setup" except choose "common" rather than "individual" at step 7.) A common scheme is one in which there is a single rating for each application — not one for each Reviewer — that any Reviewer (or, if you prefer, only any Manager) can edit.

   Labeling applicants with a characteristic

   You use the individual scheme as in the "Simple setup" described above and the common scheme to attach labels to applications. For example, suppose you are interested in whether applicants speak Spanish. Then you can create a common rating scheme with one level, with the label "Spanish". An applicant given that "rating" effectively has the label "Spanish" (or maybe just "S", to save space), attached to it, so that when evaluating applications your colleagues can easily pick out the Spanish speakers. Anyone can edit the "rating": if one Reviewer has assigned it to an applicant, but another Reviewer is sure that, despite the applicant's Spanish sounding name Celia de la Serna y Llosa, she does not speak a word of Spanish, they can change the "rating", effectively removing the label.

   Sorting applicants roughly before evaluating them in detail

   You use the individual scheme as in the "Simple setup" described above and the common scheme to sort applications roughly, perhaps on the basis of Placement Officers' recommendations. You can make the scheme editable only by Managers, who classify applications in a small number of tiers — perhaps just two ("Recommended"/"Not recommender"? "A"/"B"?). This classification is visible to Reviewers (but maybe not editable, if that's what you want), who might pay more attention to the applicants in the top tier when making their more detailed individual ratings.

If you would like us to add features to the system, please click the Feature requests link at the bottom of any page.