The first case is the VentureQuery project, which explored whether concepts and techniques of DR could be leveraged to provide an imple-mentable model of embedded explanations for a software application. As discussed earlier, the theory underlying this research is that DR-based explanations may help to make more transparent the structure of a system by exposing design team deliberations including system requirements, envisioned use scenarios, and the technical, cognitive, organizational, and other constraints applied in the development process.
The VentureQuery project involved analysis, design, and construction of a software system to create and automatically publish electronic question-naires on the web. A goal of the research design was to provide a project of realistic complexity to act as a source for DR, and to capture and structure the DR in a system capable of providing it back to system users. The team decided that the target application would consist of a web-based question–answer system in the form of a venture capital-seeking “game”. The application was intended to help to educate novice e-business entrepreneurs in the venture capital-seeking process.
The project team consisted of 12 graduate students drawn from a Masters of Science course at a UK university. About half of the project team had significant systems development experience and all had a strong interest in the process of system design. Twenty-one meetings of the core design team were recorded on audiotape. An additional three meetings between members of the design team and various project reviewers and potential users were also recorded in full. Meetings averaged 90 min. Tapes were transcribed to text files resulting in over 400 pages of design meeting dialog. In addition to the design meeting tapes, other project artifacts were analyzed for their contribution to the DR including domain analysis documents, design documents (e.g., flip chart drawings), various Unified Modeling Language (UML) diagrams, meeting agendas and notes, and e-mail between team members.
Design meeting transcripts were analyzed using the Atlas/ti (www.atlasti.de) qualitative analysis software package. Coded transcript fragments were fairly coarse-grained to ensure that the context of a given design deliberation was not lost in analysis. Based on word counts, approximately 52% of the content of the design meetings related directly to design of the application. Design deliberations were coded, extracted, and then captured as DR in a database developed to act as a knowledge base for the systems explanation facility. The Questions, Options, Criteria (QOC), DR semi-formalism [17] was used as the representational medium for the study. This selection was made based on QOC’s balance of ease of use with representational fidelity. QOC is a relatively simple and sparse method for representing DR. This simplicity was deemed an essential trait in the context of this study, as it was felt to most closely parallel the selection criteria likely to be applied in applied project settings, where practitioners are unlikely to invest time learning a potentially more richly expressive, but necessarily more complex and difficult to use formalism.
In the QOC notation, Questions highlight issues that have been identified as relevant to the design, Options are the potential solution approaches that have been identified to address a given question, and Criteria are the reasons that are considered for or against each of the identified options. Whether a criterion is considered a positive or negative factor in the evaluation of a given option is represented in the links, known as Assessments, between Options and Criteria. Assessments in QOC are not assigned weights to represent their relative importance to the argument for an Option. Criteria may be instances of Metacriteria (such criteria are called bridging criteria by QOC’s designers), though this relationship is not required. Finally, Questions may be derived from Options (the Consequent Questions of QOC) as a particular design issue is discussed. In addition, the framework was elaborated with the code QOC Outline.
