Questions

When instructors add or edit a question, they arrive at the question-editing page, authortool/question.jsp. The code for question.jsp includes question.js, a set of Javascript methods that the designers moved to a different file simply to reduce the length of question.jsp. A question consists of the following parts:

• a unique ID number;
• the type of question;
• various display options;
• the question provenance (book, chapter, remarks);
• keywords describing this question;
• the question statement;
• figures (may be none);
• question data (data that ACE needs to instantiate the question or evaluate responses);
• evaluators (at least one).

ACE stores master-authored questions in questions_v3. It stores questions that have been modified or newly authored by instructors in user_questions_v3. The tables are very similar, but user_questions_v3 contains a field for the ACE ID of the instructor. If the instructor has modified a database question, ACE stores the modified question in user_questions_v3 with the same unique question ID as the original database question. If the question is a new one, ACE gives it a negative unique ID number. ACE stores figures, question data, evaluators, and (in ACE 1.6) molecules associated with the question in their own separate tables, again putting the information from database and locally authored questions in separate tables.

• PRELOAD: load the structure in Figure 1 onto the LewisJS or or MarvinJS canvas if there is no previous response
• FIG_TO_PRELOAD (two bits): which figure (up to Figure 4) to load into the response applet
• SHOWMAPPING: this question uses atom mapping
• THREEDIM: this question asks about 3D conformation and requires students to draw structures in 3D
• SHOWNOH, SHOWHETEROH, SHOWALLH: (two bits) how to show implicit H atoms
• SHOWALLC: show the letter C for every C atom
• SHOWLONEPAIRS: turn on lone pair display
• SHOWRSLABELS: label stereocenters with R and S configurations
• SHOW_COORD_BOND: display a toolbar button for the coordinate bond
• BADVALENCEINVISIBLE: don't highlight valence errors in Marvin
• USES_SUBSTNS: this question uses R groups or, in numeric questions, variable values; different students see different versions
• DISALLOW_SUPERFLUOUS_WEDGES: generate an error message when a student draws wedge bonds at nonstereocenters
• SCRAMBLE: scramble options in multiple-choice, ordering, and fill-in-the-blank questions
• EXCEPT_LAST: when scrambling options, leave the last option last
• DISALLOW_MULT: disallow more than one selection in multiple-choice questions
• ALLOW_UNRANKED: allow students not to rank some items in ordering questions
• LABEL_ORBS: allow labeling of orbitals in an orbital energy diagram
• NUMS_ONLY: require numbers for table entries
• REQUIRE_INT: require a numeric response to be an integer
• USE_SCI_NOTN: use scientific notation in a numeric question
• NO_CONSTANTS_FIELD: in equations questions, do not include a field for constants
• NO_CHEM_FORMATTING: do not format text according to organic chemistry conventions; e.g., do not convert N3^- to N₃⁻
• HIDE: hide a question from other instructors (available to master author only)

ACE displays only the options that are relevant to the selected question type. The question type has effects in various parts of ACE. For example, ACE allots more space to the Marvin canvas on the question-answering page for mechanism and synthesis questions, it makes different evaluators available when the author has turned on the SHOWMAPPING or THREEDIM flags, and it creates new figures and evaluates responses differently when the author turns on the USES_SUBSTNS flag.

A question's book may be Bruice, Wade, or Other. When a question is from Bruice or Wade, the author should enter a chapter number and question number; otherwise, the author should enter his own initials. ACE uses the chapter numbers to build the list of per-chapter questions from Bruice or Wade in the assignment assembly tool. ACE provides as options on the question-editing page all of the books associated with courses in the database. ACE displays each question's book, chapter, and remarks to instructors only in the assignment assembly tool and on the question list page of an assignment.

The question statement is simply the English text of a question, such as, "Draw the product of the following reaction." (The question statements of fill-in-the-blank questions are more complex; see below.) A question must have either a statement or at least one figure for ACE to save it.

If the question set in which a question resides has a common question statement, then, when ACE displays any question in that set to a student, it appends the common question statement to the beginning of the question's own statement. You may edit the common question statement directly from the question set display page by pressing the Edit icon next to the common question statement on the upper right, but, in ACE 1.6, you must click on the Edit topics and question sets link on the upper left of the question set display page. Furthermore, in ACE 1.6, instructors may not modify the common question statements of database question sets, though they may edit the common question statements of question sets that they themselves have created.

The author writes the question statement as ordinary text, but ACE uses Utils.toDisplay() to format it in a chemically relevant way; for example, C5H5^- becomes C₅H₅^–. ACE includes a public access page for authors to see the rules and to enter expressions to see how ACE formats them. ACE uses regular expressions to format the text; starting in ACE 1.7, ACE reads the expressions from WEB-INF/displayRules.txt. ACE uses these rules to format not only the question statement, but also feedback, question data text, text above and below the arrow in a reaction figure, and assignment names. An author may also use standard HTML commands and character entity references such as <i> and &middot; to format the question statement.

The statement of a fill-in-the-blank question includes one or more pull-down menus. The author denotes a pulldown menu in the question statement with the notation [[1, 4, 5]], in which the menu options consist of question data 1, 4, and 5.

The statement of a numeric question may contain one or more variables in the format [[x1]], etc. ACE replaces these variables with words or arithmetical values when it displays the question statement to the user.

• molecule, a single MarvinJS drawing stored in the MRV format. (Originally, ACE stored a molecule figure in MOL format. Later, we introduced the mechanism figure, which ACE stored in MRV format. Because the MRV format could store anything that MOL could, but the reverse was not true, we then eliminated the mechanism figure and began storing all molecules in MRV.) The drawing may contain a single structure, or it may include several structures or objects such as textboxes and electron-flow arrows.
• reaction, a Marvin drawing plus some text that ACE displays above and below a reaction arrow.
• synthesis, a Marvin drawing plus one or more reaction conditions associated with stages in a multistep synthesis.
• Lewis structure, a LewisJS drawing. The drawing is stored as a single text file in an extended MOL format that describes a molecular structure and the lone pairs associated with its atoms.
• Jmol, a figure that ACE should display in a Jmol applet. Most often this figure will be a depiction of orbitals. The question author uploads a single file such as one in the .mo format. The question author may also specify Jmol scripts that ACE should execute when it displays the applet.
• image, a JPEG, PNG, or other graphic file that an author uploads into ACE. Most figures in ACE of the image type are spectra.

ACE uses JChem methods to convert all figure types (except Jmol figures and images) into SVG XML that can be embedded directly into the HTML.

ACE stores most of the information about a figure in figures_v5 or user_figures_v5.

ACE stores the filenames of images in their own table, images_v2 or user_images_v2. It stores the images themselves in their own directories on the disk: figures and user_figures, respectively.

Depending on the figure type and question type, an author may be able to instruct ACE to load one of the question figures into the response collector. In the case of clickable image and draw-vector questions, ACE always displays Figure 1 as the image on which the user should click; in the case of other questions, the author can choose to load any of Figures 1–4 into the response collector.

• text for rank, multiple choice, choose-and-explain, and fill-in-the-blank questions;
• Marvin, consisting of a MarvinJS structure and its name;
• R-group collections;
• possible values for variables in the question statement;
• rules about permissible starting materials for multistep syntheses;
• table dimensions and captions;
• table prefill values;
• orbital energy diagram dimensions, column captions, and an energy unit and scale for the y-axis;
• reaction coordinate diagram dimensions, labels for the minima and maxima, and an energy unit and scale for the y-axis;
• color for the mark in a clickable image question;
• color for the arrows in a draw-vectors question;
• additional acceptable words in a logical statements question;
• equations to preload in an equations question.

The question-editing page will show a section for question data only if the question type requires it.

Multiple-choice and fill-in-the-blank questions require two or more options from which students will choose a response, and ordering/ranking questions require two or more items for students to sequence. The question author enters these options or items as question data, which may be of the Marvin or text types. The author also enters a name for question data of the Marvin type.

Questions that use R groups require one or more question data of the R-group type. In an R-group question, Figure 1 contains one or more R¹, R², etc., and each student will see a different group substituted for each Rⁿ. Each question datum n defines the groups that ACE may substitute for Rⁿ in the figure. When an author adds or edits an R-group question datum, the page that appears, authortool/qData/loadQData.jsp, lists a set of R-group collections, each defined by a comma-separated list of shortcut group abbreviations, such as Me, Ph, etc., that themselves are defined in a shortcut group definition file, ../../marvin-extra/ACE.abbrevgroups. For example, one R-group collection consists of alkyl groups with one to four C atoms, one contains aryl groups, etc. The author may select one or more of these R-group collections for each R group in the question figure. A master author can also add or edit R-group collections from this page.

Numeric questions may contain variables in the question statement in the format [[x1]], [[x2]], etc. These questions require as many question data as there are variables. Each question datum contains a list of arithmetical values, or pairs of words and arithmetical values, that may substitute for each variable. Each student will see a different word or arithmetical value substituted for each x_n in the question statement, and the author may use the corresponding arithmetical value in the evaluator that assesses the accuracy of a student's numerical response.

Complete-the-table questions require the author to specify in two items the number of rows and columns that the table should contain. The author may also specify row and column captions in these two items and, in a third item, prefill values for the table. These items are all of the text type.

Orbital energy diagram questions require the author to specify in one item the number of rows that the diagram should contain. The author may also specify captions for two of the three columns. This item is of the text type.

Reaction coordinate diagram questions require the author to specify in one item the number of rows and columns that the diagram should contain. The author may also specify labels for the maxima and minima. This item is of the text type.

Multistep synthesis questions require that the author add at least one rule describing the permissible starting materials for each question. An author may require permissible starting materials to be particular compounds, to have one or more particular functional groups, to have a certain number of contiguous or total C atoms, to have a certain formula, to have a certain number of rings, or to be uncharged and metal-free. Authors may combine rules in any logical combination. (By default, ACE combines them with OR conjunctions.) The evaluators SynthScheme and SynthEfficiency require the permissible starting materials to evaluate a response.

An Evaluator in ACE contains a list of one or more Subevaluators, plus a grade and feedback that ACE will assign to the student if the student's response satisfies this evaluator. If an Evaluator's list of Subevaluators has more than one member, the Evaluator also contains an expression that describes how to combine the Subevaluators logically. Each Subevaluator contains the information that ACE needs to instantiate one of the approximately 75 evaluator implementations, which exercise the actual calculations on the students' responses.

When an author creates an evaluator, the evaluator is, by default, a simple one, consisting of an Evaluator that contains just one Subevaluator. The author may combine any number of simple evaluators into a complex evaluator; in this case, ACE subsumes the Subevaluators of the second and subsequent simple evaluators into the list of Subevaluators inside the first Evaluator. Conversely, an author may split a joined complex evaluator into its constituent simple evaluators; here, ACE removes each Subevaluator but one from the joined Evaluator and places each removed Subevaluator into its own new Evaluator.

ACE currently has over 75 different types of evaluator implementations, all of which reside in evals/impl/. We have divided our evaluators into three groups: evaluators for chemistry, generic, and math & physics questions. Some evaluators are appropriate for several different question types within each class (e.g., ACE can use Is to evaluate skeletal, conformation, atom mapping, mechanism, and synthesis questions, and it can use MultipleCheck for both multiple-choice and fill-in-the-blank questions), whereas others apply to only a single question type (e.g., ACE can use RankOrderCheck only to evaluate ordering/ranking questions). ACE makes available to authors only the evaluators appropriate to the question's type.

When an author adds or edits a simple evaluator, the page that appears, authortool/evaluators/loadEvaluatorChem.jsp, loadEvaluatorGeneric.jsp, or loadEvaluatorPhysics.jsp, consists of three parts. The upper part, derived from selectorFooter.jsp.h, allows the author to select an evaluator type appropriate for the question type and the points associated with a response that satisfies the evaluator. The lower part, also derived from selectorFooter.jsp.h, allows the author to enter the feedback associated with this evaluator. ACE will suggest feedback for some evaluator types. ACE will also generate automatic feedback for some evaluator types, as shown in the table; ACE stores these types in a List in EvalManager. The middle part is evaluator-specific and may contain LewisJS or MarvinJS, checkboxes and pulldown menus, etc. If a subevaluator of a complex evaluator generates automatic feedback, then the complex evaluator will return that automatic feedback if the subevaluator is the last individual expression of an OR expression or the first individual expression of an AND expression.

For most question types, it is possible for an author to write an evaluator that examines whether a response is correct. However, no single evaluator exists that can determine whether a mechanism or synthesis is correct. Instead, ACE must evaluate responses to these questions by seeing whether the response contains any mistakes. For example, the response to a mechanism question must be wrong if it does not arrive at the target compound, if it uses starting materials that the question description did not provide, if it includes or omits certain key intermediates, if the electron-flow arrows in any step do not lead to the compounds in the subsequent step, and so on. On the other hand, if a response fails to satisfy any of these evaluators, then the response contains no errors, so it must be correct. We use the evaluator, "If the number of C atoms is more than 0," to catch any response that has failed to satisfy any of the other evaluators, and we associate this evaluator with a grade of 1.0 and the feedback, "Good! Your mechanism appears to be correct."

Additional information that evaluators use

Below the evaluators, a master author may see buttons that allow him or her to add or modify (but not delete) definitions of functional groups, reaction conditions, impossible starting materials in synthesis questions, compounds that must be pulled from the reagents menu in synthesis questions, and unit conversion factors. A button to edit a table appears only if the author is editing a question of a type that uses evaluators that use the information in that particular table.

A word about reaction condition definitions. We use ChemAxon's Reactor to calculate reaction products from starting materials that the user provides and reaction definitions that we provide. Because each reaction definition defines one reaction, but ACE asks students to subject starting materials and intermediates to reaction conditions under which many different reactions may occur, we combine and modify definitions using our own terms as well.

Preview

When authors make changes to a question, they can examine the way that the change affects ACE's evaluation of various responses by pressing the Preview button. ACE directs the author to authortool/startPreview.jsp, then to homework/jumpGo.jsp, and thence to the question-answering page, homework/answerframe.jsp.

An author can see all of the responses that students have made to a particular question by pressing the View Responses button. The page that appears, authortool/FIHistory.jsp, shows which evaluators the 20 most frequent responses satisfied. An author may choose to add or edit an evaluator based on this information. ACE will collate the responses of all students, not just that author's students, who have responded to this question. (Locally authored questions, of course, are accessible only to the students of the author who created those questions.) The View Responses function does not work for mechanism or synthesis questions because ACE has no algorithm for deciding whether two responses are identical.