Advanced Assessment Editor

The advanced assessment editor uses a simple, text-based format to create questions. Edit questions as you would on paper. See the following multiple-choice question where the correct answer is 'c':

Type: MC
1) What is the capital of Sweden?
a. Oslo
b. Helsinki
*c. Stockholm

In addition to the question text, questions can also have properties associated with them. The properties appear above the question number. The first property must be the question type. See Reference section below for supported question types.

We describe other properties in the sections that follow.

Preview Questions

The editor includes a 'Preview Questions' button that you can click to preview the exam questions as the student would see them. For your convenience the correct answer is indicated in multiple-choice and multiple-answer questions. For fill-in-the-blank questions, you can click a 'Check Answer' button to evaluate your answer. 'Check Answer' is also helpful for questions that use math or regular expressions.

You can paste the examples from this help file into the editor and click 'Preview Questions' to see the questions. You can selectively preview questions by selecting the question text in the editor before clicking 'Preview Questions.'

Import Questions

You can import questions from other formats (including QTI) into the editor by clicking 'Import Questions' and specifying an import file. After uploading the file, the editor inserts the new questions after any existing questions. You can also import QTI .zip files that contain questions with associated resources, such as images. The import stores resources in the course's Resources folder, and the questions refer to the course resources. Note that any existing course resource with the same name is overwritten. After importing, you can validate, preview, and edit the questions before saving.


Generally, question text is plain, unformatted text with optional special codes that indicate question structures and advanced formatting. We describe the special codes below.

Fill-in-the-blank and inline choices

You define the blank in a fill-in-the-blank question by entering a minimum of three sequential underscore (___) characters in the question text. To make the blank longer, specify more than three underscores. Each underscore widens the blank by one character width.

Type: F
1) The capital of Sweden, __________, is also known as Venice of the North.
a. Stockholm

You define the drop-down list location in "inline" multiple-choice and matching questions by entering four bracket-enclosed hyphens: [----]. The assessment player automatically determines the correct display width for the drop-down list.

Type: MC
Options: Inline
1) Michigan is located in the [----] region of the country.
a. east
b. south
*c. midwest
d. southeast
e. west

You enter equations in either ASCII Math or LaTeX format. Enclose ASCII Math equations in back quotes (``), and enclose LaTeX equations in back quote and dollar (`$$`).

Type: F
1) The chemical `H_2O` is commonly called ______.
a. water

You can include course media files such as video, images, and audio in your question with the bracket-enclosed [Media] tag. (Course media files are stored in the course media library.) You can double click on a resource in the Media Library list to insert the Media tag for that resource.

Type: MC
1) Which of these is the title song for a recent musical?
*a). [Media: mammamia.mp3]
b). [Media: dancingqueen.mp3]

The following media types are supported:

Video.flvAdobe Flash
Audio.mp3MPEG Audio Stream, Layer III
Image.gif,.jpg,.jpeg,.jfif,.pngGraphic Interchange Format, JPEG/JIFF Image, and Portable Network Graphic

A media tag can have optional title and description text specified. If the text contains a comma, bracket, quote or backslash character you have to enclose the text in quotes and escape the quote and backslash with an extra backslash.

[Media: filename, title, description]
[Media: sunset.jpg]
[Media: sunset.jpg, Sunset]
[Media: sunset.jpg, Sunset, Picture displaying the sun setting]
[Media: sunset.jpg, "A \"photo\" of a sunset"]

You can specify advanced formatting by enclosing an entire question text with the tags [HTML][/HTML]. Between the tags, you can use standard HTML formatting tags. Note that you must enclose the entire question text, not just part of it.

Type: F
Options: Number, Multiple
4) [HTML]<p>Fill in the table</p>
a. 9
b. 5
c. 4

You specify general question feedback by starting a line with the @ sign immediately after the question body. Feedback text can contain $-enclosed question variables.

Type: F
1) The capital of Sweden is _________.
@ The capital of Sweden is Stockholm
a. Stockholm

For multiple-choice questions, create choice-specific feedback (feedback shown when the choice is chosen by the student) by placing the feedback line below the choice.

Type: MC
1) What is the capital of Sweden?
*a. Stockholm
@ That's correct
b. Oslo
@ Oslo is the capital of Norway
c. Helsinki
@ Helsinki is the capital of Finland

Feedback can be displayed only if a condition is met. The condition is a bracket-enclosed expression that can contain question variables, literals, and the pre-defined variables named score, teacher, answer, and answer# (for multiple-answer questions). When multiple feedback lines exist, only feedback from the first satisified condition is displayed.

Type: F
Options: Number
Var: a = 3..9
2) 2 * $a$ = _____
@[teacher] This feedback is displayed only to teachers
@[score>=1] This feedback is displayed only when the student's question score is greater than or equal to 1
@[answer="2+$a$"] This feedback is displayed when the student added the numbers instead of multiplying them
@ This feedback is displayed only if no other feedback has been displayed.
a. 2*$a$

To save some typing, you can specify the shortcut "=" instead of "answer=" in the feedback condition. The following two questions are functionally equivalent.

Type: F
1) The capital of Sweden is _________.
@[answer="Stockholm"] Yes. The capital of Sweden is Stockholm
@[answer="Oslo"] No. Oslo is the capital of Sweden's western neighbor, Norway.
@[answer="Södertälje"] No. But Södertälje is sometimes called the car capital of Sweden.
@ Not even close
a. Stockholm

Type: F
2) The capital of Sweden is _________.
@[=Stockholm] Yes. The capital of Sweden is Stockholm
@[=Oslo] No. Oslo is the capital of Sweden's western neighbor, Norway.
@[=Södertälje] No. But Södertälje is sometimes called the car capital of Sweden.
@ Not even close
a. Stockholm

Your can define your own feedback condition shortcuts with the questionfeedbacktypes element in your domain customization XML. For example, we could define these Solution and Correct definitions in the domain customization:

<questionfeedbacktype display="Solution" group="Solution" condition="teacher"/>
<questionfeedbacktype display="Correct" condition="score>=1"/>

We can then refer to those pre-defined conditions in feedback conditions:

Type: F
Options: Number
Var: a = 3..9
2) 2 * $a$ = _____
@[Solution] Feedback that only the teachers see
@[Correct] Feedback for those with score >= 1
@[=2+$a$] Feedback for those who added the numbers instead of multiplying them
@ Feedback that is displayed only if no other feedback has been displayed 
a. 2*$a$

When multiple feedback rows exist, BrainHoney displays only the first row that matches a condition. You can enable multiple-feedback-row display by grouping conditions; BrainHoney displays the first matching row from each group unless DisplayAll is specified. Start a condition with GroupName: to identify a feedback group:

Type: F
The capital of Sweden is _________.
@[GroupA:score>=1] Corrrect
@[GroupA:] Incorrect
@ This line is always displayed, even though feedback from GroupA is also displayed.
a. Stockholm

Questions that have multiple answers, such as multiple fill-in-the-blank or matching questions use the shortcut pattern "a=" to specify feedback that applies to a specific answer.

Type: MT
Score: Partial, Round
Points: 4
4) Capitals
@[GroupA:a=a] For row 'a' (Sweden) the student answered 'a' (Stockholm).
@[GroupA:a=b] For row 'a' (Sweden) the student answered 'b' (Oslo).
@[GroupB:b<>b] For row 'b' (Norway) the student did not answer 'b' (Oslo).
@[GroupC:c=a] For row 'c' (Finland) the student answered 'a' (Stockholm).
a. Sweden = Stockholm
b. Norway = Oslo
c. Finland = Helsinki
d. Denmark = Copenhagen

You can use answercorrect variable in feedback conditions to define specific feedbacks for each blank in a multiple-fill-in-the-blank question. Use displayall to display all feedbacks with matching conditions within a group.

Type: F
Options: Multiple
Score: Partial, Round
Points: 4
@[GroupA:displayall:answercorrect1=false] Your answer to first blank is incorrect.
@[GroupA:displayall:answercorrect2=false] You answer to second blank is incorrect.
1) _____ Jefferson was the ______ President of the US.
a. Thomas
b. 3rd
Partial score

For the multiple-choice and fill-in-the-blank questions, you can specify a score for individual answer response or a correct answer. You place the score line below the multiple-choice question choices or fill-in-the-blank answers, and start the line with %. Scores are relative values comparing to the question score. See Choice Score Attribute and Answer Value Score Attribute in the Question Schema for details.

Type: MC
Score: Partial, Round
1) What is the capital of Sweden?
*a. Stockholm
% 1
b. Oslo
% 0.5
c. Helsinki
% 0.1
z. None of the above

Type: F
Options: Multiple
Score: Partial, Round
Points: 4
1) _____ Jefferson was the ______ President of the US.
a. Thomas
% 0.5|0.5|0.5
b. 3rd
% 0.5|0.5|0.5|0.4
Response labels

For multiple-choice or multiple-answer questions, you can specify the type and format of labels for answer responses. See Label Attribute for the type format.

Type: MC
Label: A.
1) What is the capital of Sweden?
*a. Stockholm
b. Oslo
c. Helsinki
z. None of the above
Question Options
Keep response last

To keep a given response last when randomizing responses, use the letter 'z' for the last response.

Type: MC
1) What is the capital of Sweden?
*a. Stockholm
b. Oslo
c. Helsinki
z. None of the above
Maintain response order

Use the letter 'x' so the position of a response will remain the same while the order of the remaining responses can be randomized. Responses with the letter 'z' will be placed before any last reponses with the letter 'x' if there are any.

Type: MC
Label: a.
1) Which colors are primary?
x. Red
b. Green
x. Blue
d. Pink
*e. both a and c
Inline multiple-choice questions

You can change the display of a multiple-choice question to use a drop-down list rather than bubble choices by specifying the "Inline" option and the location of the drop-down list with [----]. Response choices must be a single line of plain text.

Type: MC
Options: Inline
1) The capital of Sweden is [----].
*a. Stockholm
b. Oslo
c. Helsinki
z. None of the above
Matching questions

Matching questions must have a left- and a right-hand side of each match. Use the equal sign (=) to separate the two sides. You can leave the left side empty to create extra match items in the right column of choices. To format a choice as HTML, enclose both sides of the match with the [HTML][/HTML] tags.

Type: MT
Options: Inline
1) Match the animals with their sounds.
a. dog = woof
b. cat = meow
c. bird = chirp
d. = moo

You can also create drop-down boxes within the question text by specifying the "Inline" option above the question and drop-down list locations [----] in the question text. In this mode the match left-hand side is not displayed, but acts as the key to which box is the right answer. 1 = first, 2 = second, and so forth. In this mode you can also use the "MaintainOrder" option to preserve the choice order in the drop-down

Type: MT
Options: Inline, MaintainOrder
1) Indicate the correct animal sounds: 
The dog [----] chased the cat [----] that chased the bird [----].
a. 1 = woof
b. 3 = chirp
c. 2 = meow
d. = moo
Alternative fill-in-the-blank answers and case sensitivity

A fill-in-the-blank question can have more than one correct answer. Specify multiple correct answers to a fill-in-the-blank-question as choices a, b, c, and so forth. To accept case-insensitive answers, specify the "IgnoreCase" option above the question.

Type: F
Options: IgnoreCase
1) ___________________ was the 16th president of the United States.
a. Abraham Lincoln
b. Abe Lincoln
c. Lincoln
Multiple blanks in fill-in-the-blank questions

Specify multiple blanks in a fill-in-the-blank question by specifying the "Multiple" option above the question and multiple blanks in the question text. To specify multiple correct answers to a blank, put each correct answer on a different line.

Type: F
Options: Multiple
1) The _____ dog chased the ____ cat.
a. barking
b. meowing
Show correct answers

Specify show correct answers in a fill-in-the-blank question by specifying the "ShowCorrect" option above the question and multiple blanks in the question text. Correct answers will be displayed when the question is reviewed if the display feedback setting is enabled.

Type: F
Options: ShowCorrect
1) The _____ dog chased the ____ cat.
a. barking
b. meowing
Show work

For any question type except essay, you can set the "Workspace" option to give the student an area to show their work. If a question has the "Workspace" option and a question is answered incorrectly, the automated grading assigns a 0 score for the question and forwards the question to the teacher who can manually override the auto-assigned score.

Type: MC
Options: Workspace
1) What is `(a+b)^2` if a = 3 and b = 2? Show your work.
a. 20
*b. 25
c. 13
Composite questions

Sometimes a set of question will have a common header or variables. Use a composite question to share variables and to ensure that the header is shared by keeping the question together and on one page. The questions that are part of the composite questions must have a question number with the format X.Y where is X is the composite question number and Y is number of the question part.

Type: Composite
Options: MaintainOrder
Var: start = 2..15
1) For the following three questions start with the number $start$ and apply 
   the operations in order.

Type: F
Options: Number
1.1) Add 3: ________
a) $start$+3

Type: F
Options: Number
1.2) Now multiply by 5  ________
a) ($start$+3)*5

Type: F
Options: Number
1.3) Now subtract 11 _______
a) ($start$+3)*5-11
Custom questions

A content developer can develop custom questions. To use a custom question specify the Url, resource path, or custom question name as defined in <customquestions> on the Custom property. If the question emulates a standard question specify the question type and answers like that question. For custom grading use the question type 'Custom'.

Type: MC
Custom: TrueFalse.xml
Var: odd = 3..20,2
1) $odd$ is an odd number
*a. True
z. False

Type: Custom
Var: a = 6..20
Var: b = 6..20
3) $a$ * $b$ = ______
Math and Science Questions

You can click symbols on the Equations panel to enter the symbol at the editor's current cursor location. You can also type the symbol codes directly into the editor.

ASCII MathLaTeXOutput

You can define variables in a question to generate data-driven questions by using the "Var" property. Variables are most useful in fill-in-the-blank questions of type Number, Match, and Equivalent. You can define answers as mathematical formulas using the variables. To define a range variable you specify the variable name, start and end of the range, and an optional step value. For example, Var: a = 3..9,2 defines the variable "a" to be 3, 5, 7, or 9. You refer to a variable by enclosing the variable name in dollar signs ($).

Type: F
Options: Number
Var: a = 4..9
Var: b = 3..9,2
1) $a$ * $b$ = _____
a) $a$*$b$

You can also specify a variable as a comma-separated list of values. Lists can contain text or numbers. Use quotes to include a comma in your variable value. You can group list variables by defining variable names as groupname.variablename. From each group variable, the question substitutes a value from the same random index into the question. In the following example, the "unit" group variables have three possible values. If the question uses km/h from unit.text, then it uses 1000/3600 from unit.conversion. Similarly, if it uses mph from unit.text, then it uses 0.44704 from unit.conversion.

Type: F
Options: Number
Var: speed = 2..50
Var: unit.text = km/h,ft/s,mph
Var: unit.conversion = 1000/3600,0.3048,0.44704
1) Convert to m/s: $speed$ $unit.text$ = ________ m/s
a) $speed$*$unit.conversion$
Fill-in-the-blank evaluation

Fill-in-the-blank questions by default evaluate as correct if the entered answer text exactly matches the keyed answer text. For math and science questions you can specify one of these types for more flexible evaluation:

NumberEvaluates the answer as a number. Numerically compares the answer to the correct answer. Any number within 1% of the correct answer is considered correct. You can specify your own match range if you use two dots between the start and the end value of the correct answer. For example: 3.14..3.1416You can also specify the required number of significant figures with the "Figures" property. Students must enter exactly the number of significant figures to get the question correct.
MatchCompares the user-entered expression to the correct expression by matching a normalized expression parse tree. Use "Match" for simplification type questions. For example if the answer is 2a, a+a is not correct.Please see Supported operations below for available operations.
EquivalentEvaluates the user-entered expression as correct if it is mathematically equivalent to the answer. For example, if the answer is 2a, a+a is correct.Correctness is computed by evaluating the expressions using sample values. The current sample values are: 0.123456789012, 0.345678901234, 0.890123456789, 5.678901234, -0.7654321. You can change the domain of these values for you question by creating a list variable named "domain".
Supported operations
+ Addition1+2
- Subtraction1-2 or -4
* Multiplication3*x or 3x
/ Division3/4
^ exponentialx^2
pi 3.141592pi
e scientific notation1.6e26
|x| or abs(x) absolute value|-5| = 5 or abs(-3) = 3
sin, cos, tan, sec, csc, cot, asin, acos, atan (angle in radians)cos(pi) = -1
sqrt square rootsqrt(y)
x^(1/n) for the n root of a number x^(1/3)
ln, log natual log and log base 10ln(x)
eval evaluate variable expressioneval($a$ + $b$)
Expression evaluation for Match math questions

When creating expressions for Match questions that use variables it may be necessary to evaluate part of the expression. Using the built-in eval function in the expression resolves the parameters. Note that you do not need to use eval for Number of Equivalent questions, as these types are automatically evaluated.

Type: F
Options: Match
Var: a = 2..9
Var: b = 2..9
Var: c = 2..9
1) Simplify $a$($b$x+$c$)
a) eval($a$*$b$)x+eval($a$*$c$)
Expression evaluation for text

In the question, choice, or feedback text, you can control the display format of any computed number or expression by using the $eval(expression,format) syntax, where expression is the numerical expression to be evaluated, and format is the output format of the number.

Type: MC
Var: a = 12..19
2) What is $a$/2?
*a. $eval($a$/2,#.0)
b. $eval(($a$-1)/2,#.0)
c. $eval(($a$+1)/2,#.0)
Question banks and links

In addition to creating new questions, you can also use existing questions in your assessment. You can link to a specific question or a group of questions.


To link to a specific question use the question type Link and the question id as the id. The easiest way to link to a question is to navigate to it in the Questions Banks panel on the right side and double click the question. Referrer displays the entity id and item id to identify where the link comes from or authorizes access to the question.

For your convenience, the editor shows the question title by the number, and the full question definition enclosed in the tags [LINKEDQUESTION][/LINKEDQUESTION]. You cannot edit the title nor the linked question from the editor.

Type: Link
ID: f6ab5583a4904129a24807cca70a985a
4) What is the capital of Sweden?
Type: MC
1) What is the capital of Sweden?
a. Oslo
b. Helsinki
*c. Stockholm
Bank with item

You can link to an entire assessment by using the Bank type and specifying the item id as the id. Optionally you can specify the max number of questions by using the Count option. Use the Question Banks panel to choose the item. The text by the number is the title of the item you are linking to. You cannot edit the title.

Type: Bank
Count: 5
5) Simple Algebra
Bank with query

You can link to a subset of all questions in a course by using the Bank type and a query. You should also specify a count of questions to use. The text by the number is the course title.

Type: Bank
Query: Type = F and Option = Number and Objective = 'MA.7-3'
Count: 5
6) Algebra 1
Using questions in other course

You can use each of the options above pointing to other courses. Prefix the id with COURSEID: where COURSEID is the internal id of the target course. The easiest way of creating links to other courses is to click the More Banks button in the Question Banks panel.


The complete syntax for the assessment editor follows. All properties are optional unless otherwise noted. Parentheses and vertical bars '(|)' indicate a set of mutual-exclusive items. Square brackets '[' indicate optional items. The plus sign '+' indicates the item may be repeated. All properties containing comma seperated list, such as Groups, Objectives, and Var, may contain commas in their values by surrounding the value with quotes.

Type: (MC|MA|MT|F|E|Composite|Custom|Bank|Link)
Options: [(IgnoreCase|Number|Match|Equivalent|Regex)]
Score: [POINTS][,Partial[,Round]]
Id: ID
Query: QUERY
Referrer: REFERRER
Count: COUNT

TypeQuestion type
MCMultiple choice
MAMultiple answer
FFill in the blank
CompositeComposite question
CustomCustom question
BankLink to a bank of question
LinkLink to a question defined in another bank
OptionsVarious question options
IgnoreCaseIgnore case when checking answer.
NumberTreat answer as number.
MatchUse math expression-tree matching to determine if an answer is correct.
EquivalentUse numeric substitution to determine if an answer is functionally equivalent.
RegexUse a regular expression to check if an answer is correct. See Regular Expression Syntax.
WorkspaceCreate a workspace to allow students to show their work.
MaintainOrderDo not reorder the responses, even when the exam settings allow it.
ExtraCreditQuestion counts as extra credit.
InlineUse the inline (drop-down list) view of multiple-choice and matching questions.
MultipleQuestion contains multiple blanks.
ScorePOINTSNumber of points
PartialAllow partial credit for question.
RoundIf partial credit is given, round down to the nearest whole number.
IdIDRequired property if the question type is Link, optional if Bank. It is the ID of the question or bank to use. Questions IDs have the syntax [entityid:]questionid, where entityid is optional. If not present the current course is used.
QueryStringQuery to restricts questions if question type is Bank. See XML Query for the query syntax. In addition you can use the following common properties in your query:
TypeMC MA MT F E Composite CustomQuestion type
OptionNormal IgnoreCase Number Match Equivalent RegexQuestion option
Group'GROUPNAME'Group name
Objective'Id'Learning objective
Meta-NAME'metavalue'Meta information
ReferrerID Required property if the question type is Link. It has the syntax of entityid:itemid, which indicates where the question comes from or authroizes the access to the question after the link is saved. See Item Data.
CustomUrlUrl to renderer for custom question. You can use a course resource to render by custom question by specifying the path to the resource.
CountCOUNTNumber of questions to include from a bank
GroupsGROUPNAMEComma-separated list of group names
ObjectivesOBJECTIVEIDComma-separated list of objective ids
Passage[ENTITYID:]QUESTIONIDLink to a composite question that acts as the passage for this question. Entityid is optional. If not present the current course is used.
RubricRUBRICPATHPath to the resource in this course containing the grading rubric for this essay question
HeightHEIGHTHeight of the essay area.
VarNAME = START..END[,STEP]Range variable definition. Variable is between START and END with an optional STEP.
Var[GROUP.]NAME = VALUE[,VALUE]+List variable definition. Variable is one of comma-separated list of values. Optionally the variable can have a coordination group name.
FiguresSIGFIGURESNumber of significant figures that must be in the answer.
Meta-NAMEVALUESpecify the VALUE of the meta information NAME. For example, to specify the author as John Smith type:
Meta-author: John Smith
1)BODYText of the question
@[[GROUPNAME:][DISPLAYALL:][CONDITION]]FEEDBACKAutomatic question feedback. A question may include multiple feedback rows. Feedback rows are grouped according to GROUPNAME. The first matching condition in a group is displayed unless DISPLAYALL is specified. Empty conditions are always true. So in the simplest case feedback that always displays can be created as @.
See the DLAP documentation for full syntax rules. The assessment editor supports these shortcuts:
  • FEEDBACKTYPE - Use the group and condition of a pre-defined feedback type.
  • =ANSWER - The answer equals ANSWER. For multiple choice questions ANSWER is the choice letter. Shortcut for answer=ANSWER
  • a=ANSWER - A matching or multiple inline answer mathces the answer. Shortcut for answer#=ANSWER
[*]a.CHOICEBODY For multiple-choice and multiple-answer questions this is the text of the choice. On optional star '*' indicates that the answer is correct. For fill-in-the-blank questions this is the actual answer. To specify multiple correct answers per blank, put each answer on a different line. For matching questions this should be in the form LEFT = RIGHT. For essay question, this is a sample essay to present to the teacher for grading.
@CHOICEFEEDBACKChoice-specific feedback for multiple-choice and multiple-answer questions.
LINKEDQUESTIONQUESTIONIf the question type is Link, optionally displays the definition of the linked question enclosed with tags [LINKEDQUESTION][/LINKEDQUESTION].