A number of challenges and research opportunities emerge from consideration of the findings from these three studies. Results from the Venture-Query case study highlight the difficulties associated with capturing complete DR when design activities occur not only in formal meetings, but also in informal and individual forums. If completeness is a critical attribute of the DR for a system, such as when it is used to provide an explanatory knowledge base, then design knowledge capture is one of the most pressing challenges for research. Design is a ubiquitous activity and can happen as often in the mind of a single individual riding on the train as it does in more formal contexts where it is amenable to capture. The challenge of pervasive design capture raises many questions to occupy researchers.
Our experience on the VentureQuery project also suggests that adoption of a DR process, in addition to notations and supporting tools, may help ensure capture of a more complete design knowledge base. Process prescriptions for experienced designers are, however, notoriously difficult to enforce as they are seen as disempowering these creative individuals. Explorations into better ways of integrating DR techniques into the day-to-day work of designers may help ease this problem, as might better tool support for DR capture.
Our work with the Marine Corps on antiterrorism planning decision models and tools has suggested a role for DR, in the form of scenarios, claims, and related systems artifacts, as a vehicle to facilitate transfer of technology between organizations. This work is ongoing and our results are tentative but experiences with antiterrorism planners in the field suggests the potential utility of DR, in the form of scenarios and claims analyses, as explanatory transfer packages to help prospective system adopters evaluate new technologies. Questions have already emerged about the form such a package should take, and the kinds of DR that are most useful for technology adopters in different roles and contexts.
Our work with the Marine Corps PM LAV on complex, distributed system evaluation suggests that there may be valuable linkages to be developed between evaluation techniques and tools and those used for DR. Developing techniques to relate DR to evaluation, then forward to redesign and subsequent evaluation in a cycle of learning and artifact improvement may be one way to achieve a truly progressive systems design science. Still under-researched are the downstream consequences of design decisions made when a system is still an abstract model as unrealized in working software. Repositories of DR that provide a longitudinal view of design deliberations and their consequences may help us better understand the effectiveness of the different design methodologies and tools created to support the systems design process.
It is not expected that the cases presented here and the lessons learned from their analysis will apply to all settings in which DR and design capture are attempted. In particular, situations less contrived than the ‘zoo’ study reported in the first case, and less structured and formal than in the second and third, may exhibit very different characteristics and outcomes. Still, empirical studies of DR are lacking and it is hoped that these cases can contribute to the evolving base of experiences with DR in both controlled and field settings.
