|
|
|
|
Important notice: The CSA web site was re-designed in August of 2010. Some documents then available were considered complete and static; so they were not included in the re-design and were not updated. This is one of those documents. Information about dates of posting and revision remains here, but there will be no revision of any kind after August, 2010.
| Contents | |
|---|---|
How does one evaluate a CAD model? There is no simple answer to that question, but anyone who builds a CAD model must ask how good that model is. Only by asking the question, after all, will the model maker begin to answer it. Furthermore, anyone using a CAD model will need to evaluate it as a resource either implicitly, as a user, or explicitly as a reviewer for accuracy, reliability, completeness, and accessibility.
The evaluation process must begin with the project's own description of its aims and purposes. One must start there, because the aims and purposes will certainly have informed the project methods and procedures. As it would be unfair to criticize a project to create a city street plan for omitting 3D views, so it would be improper to commend a 2D model created to explore complex structural relationships among the roofing members. Evaluation of methods and procedures especially will begin with a carefully examination of the project aims.
The most obvious concrete evaluation criterion for a model is the accuracy if the model geometry. The model maker can check geometry by comparing known dimensions in the real world to those retrieved from the model. While that is a tedious process, it is relatively innocuous if checking is done as the model is being built. Users and reviewers will find that to be more difficult, if not impossible.
If total stations are used to provide data and the coordinates are transmitted electronically from a data recorder, checking the geometry is less necessary for the model builder, but careful checks of the data transmission processes should be regularly made. Since electronic transmission eliminates hand entry of data, the opportunity for introducing error in the data entry process is greatly reduced. Individual errors are very unlikely, but systemic errors are possible, making checks of the system critical.
Both model builders and users/reviewers will find that the best test for the geometry is the appearance of the model. If everything is geometrically correct, the model should appear to be correct from any angle and with any selection of layers showing. Of course, that test is not very systematic, but it is nonetheless effective -- assuming that the model in question is either a solid or a surface model. In fact, that test is another reason to use solids or surfaces. Wire-frame models are virtually impossible to test visually. Two-dimensional models may show errors when viewed with different layer selections, but, of course, changing the viewpoint is of no value.
Evaluating the layer-naming is not difficult, though the evaluation is a subjective one. The system is effective for the model builder, quite simply, if it makes building the model easier by improving navigation and data input.
Both the model builder and subsequent users/reviewers will need to evaluate the layer-naming system to be sure it provides appropriate analytic possibilities. The potential to analyze the model is an unending requirement; the layer-naming system must make that possible. The model maker is, quite naturally, unlikely to recognize the shortcomings, if any, of a system he or she developed and used; so evaluating it may be difficult. For that reason, the model maker's evaluation of the layer-naming system should involve other scholars who may want to use the model. The most important question will be a simple one: Are there analytic categories of importance that are not treated in the layer-naming system? (If there are extra categories, that is of little importance. Extra categories may make the system slightly more cumbersome to use, but they will not limit analysis in any way.) If there are missing analytic categories, it may be necessary to reconsider the system. Since changing the layer-naming system is very difficult if the model is complete, the system should be critiqued as early in the model-making process as possible.
Secondary users of the model will also ask the crucial question: Are there analytic categories of importance that are not treated in the layer-naming system? A negative answer for those secondary users will not result in changing the layer-naming scheme but will lower the evaluation.
In addition to the layer-naming system, model makers may provide ways to access specific groups of layers; layer-grouping processes may be built into the model or provided via routines in macros or external files. Accessing specific groups of layers -- for instance all those of a specific phase -- would then be quicker and less complicated both for those who make the model and those who use or review it, making it easier to use the model effectively. A user or reviewer may therefore consider prepared mechanisms to access layer groups to be a valuable addition. The model maker may also find the model easier to use with layer-finding aids, and he or she will want to consider the added value as related to the cost of including it. (Note that ways to provide access to layer groups will probably not survive changing the model from any given CAD file format to the format used by a competing program.)
Linkage to notes and data tables should also be evaluated. In these cases, the evaluation is very simple for either model builders or secondary users. Are the links there? Are they clear and unambiguous, with differences between notes and data-table links obvious? Can the links be navigated either from the model or from the data file?
CAD operators may add other features to a model to make it more useful. Macros or other ways to manipulate the model may, for instance, provide a sequence of related views, leading the user through time or through a construction process. Photographs might also be linked to the model via a process similar to that used for data linking. Any such added features should be evaluated by secondary users; though they only add convenience. Those who have built such models will want to evaluate added features of this sort somewhat differently, since the gains must be weighed against the costs incurred.
Once the model and associated files have been completed, they should be preserved so that they can be accessed and used by other scholars. Preservation is the last and simplest part of the process, but it is also the least often completed.
Preservation is not something that can be done by the model maker, because these kinds of scholarly materials should be preserved for far longer than the life of any individual scholar. Therefore, an archival repository that is prepared to accept digital files and that is staffed with discipline-specific personnel must be found and utilized. The Archaeology Data Service (http://ads.ahds.ac.uk/) in England is one such archival repository; the Archaeological Research Institute (http://archaeology.la.asu.edu/) at Arizona State University is another.
The chosen repository will have its own deposit requirements regarding the data, the documentation , the indexing/searching information required, and so on. All those requirements should be checked in advance so that there are no surprises when the project is complete.
Archival repositories will perform the important work of preserving data files via two processes. The first, called data refresh, is fairly simple; since digital files can decay over time, they must be re-written on a scheduled basis to make certain that the original is preserved. The more difficult and more important task is that of changing file formats in response to changing computer standards and requirements. As computer hardware and software evolve, file formats change. As a result, digital files can become useless in the sense that they cannot be used by current software. This is not a new problem, but it requires considerable vigilance on the part of the archival repositories to prevent files from becoming obsolete.
The requirement that file formats be changed in response to changing computer standards -- the process is called file migration -- suggests that specialized repositories are required. Traditional archival repositories have concentrated on preserving their holdings and trying to keep them from changing. Digital archival repositories must concentrate instead on systematic changes to their content. They must have personnel with the technical skills required to accomplish file migrations as well as the discipline-specific skills required to guarantee that the migration process preserves the scholarly information in the files. For these reasons, archival repositories that specialize in digital files and the appropriate discipline are to be preferred.
As noted above, preservation is not regularly accomplished. In part, this is the result of the normal slowing of any project toward its end. When publication has been finished, it seems the project is complete, and, until recently, there has been little awareness of the need actively to preserve digital files as a final step. In part, the absence of archival preservation also reflects cost issues. The need for digital preservation has only recently been recognized; so associated costs have not been anticipated and have therefore not been built into project budgets at the outset. Raising the funds after the fact is nearly impossible. Another factor affecting archaeological projects is the well-known but little-acknowledged reluctance of archaeologists to provide full access to excavation data. Whether that reluctance derives from a fear of the results of permitting access, an unwillingness to spend additional time on the least interesting work of the project, or some other factor is not important. Finally, there have been many comments throughout this work about the need to document the model and the accompanying data files. That documentation takes time, personnel, and money. If the documentation has not been carried out as the project developed, it takes far more of those precious resources, more than most project directors are prepared to spend, to complete the documentation so that archiving can follow.
The foregoing implies some obvious needs for any scholarly project. First, archival preservation must be taken seriously from the very beginning. An archival repository should be contacted, and requirements for documentation should be clear before the project begins. Preservation costs should be determined, to the extent possible, before the project begins, and those costs should be built into the budget. Cost estimates must be realistic, and the funds must be segregated so that there is no danger that the money will not be available at the proper time. In addition, the documentation process that has so often been mentioned here must be continuous throughout the life of the project so that it is not an insuperable burden at the end.
Implicit in this discussion of archival preservation is an emphasis on the data from a project rather than the publication that derives from the data. Publication demands a selection from the total information package and a synthesis. It remains the scholar's responsibility (and well beyond the scope of this work), but providing access to the initial data is an equal responsibility, particularly when public or quasi-public money has been used to fund the project. Providing long-term access requires depositing the data files in an archival repository, and that, in turn, requires adequate preparations.
A good CAD model requires good data and data-gathering processes, proper model-building procedures, a good layer-naming system, careful and complete documentation, and archival preservation. Each of those requirements is truly critical, and the model's value will be damaged if any of them is not met. It is therefore necessary to be more than simply careful -- to be self-conscious and mindful of the importance of each piece of the puzzle from the inception of the project to the final step of archival preservation. That final archival step is required before considering the project to be complete.
The resulting CAD model is far more than a set of good drawings, more valuable indeed than any number of carefully prepared paper drawings. It is both a data source that provides unscaled geometric information and the source of a virtually infinite number of views of its subject -- produced on screen or on paper. A good CAD model is also an extraordinarily valuable analytic aid, thanks to the use of layers, and a base for future scholarship, providing the geometry of its subject and the analytic categories needed to understand it. It may also be the base to which other graphic products add additional information. Connected to data tables and/or notes, it is also part of an integrated information source about its subject. In short, a CAD model is the best way to deal with any complex 3D reality -- for the scholar in charge and for those who will one day build on that scholar's work.
| Go to glossary. | Go to Table of Contents. | Go to CSA home page. |