![]() |
Dakota Reference Manual
Version 6.15
Explore and Predict with Confidence
|
The Dakota restart utility program provides a variety of facilities for managing restart files from Dakota executions. The executable program name is dakota_restart_util
and it has the following options, as shown by the usage message returned when executing the utility without any options:
Usage: dakota_restart_util command <arg1> [<arg2> <arg3> ...] --options dakota_restart_util print <restart_file> dakota_restart_util to_neutral <restart_file> <neutral_file> dakota_restart_util from_neutral <neutral_file> <restart_file> dakota_restart_util to_tabular <restart_file> <text_file> [--custom_annotated [header] [eval_id] [interface_id]] [--output_precision <int>] dakota_restart_util remove <double> <old_restart_file> <new_restart_file> dakota_restart_util remove_ids <int_1> ... <int_n> <old_restart_file> <new_restart_file> dakota_restart_util cat <restart_file_1> ... <restart_file_n> <new_restart_file> options: --help show dakota_restart_util help message --custom_annotated arg tabular file options: header, eval_id, interface_id --freeform tabular file: freeform format --output_precision arg (=10) set tabular output precision
Several of these functions involve format conversions. In particular, the binary format used for restart files can be converted to ASCII text and printed to the screen, converted to and from a neutral file format, or converted to a tabular format for importing into 3rd-party plotting programs. In addition, a restart file with corrupted data can be repaired by value or id, and multiple restart files can be combined to create a master database.
The print
option is useful to show contents of a restart file, since the binary format is not convenient for direct inspection. The restart data is printed in full precision, so that exact matching of points is possible for restarted runs or corrupted data removals. For example, the following command
dakota_restart_util print dakota.rst
results in output similar to the following:
------------------------------------------ Restart record 1 (evaluation id 1): ------------------------------------------ Parameters: 1.8000000000000000e+00 intake_dia 1.0000000000000000e+00 flatness Active response data: Active set vector = { 3 3 3 3 } -2.4355973813420619e+00 obj_fn -4.7428486677140930e-01 nln_ineq_con_1 -4.5000000000000001e-01 nln_ineq_con_2 1.3971143170299741e-01 nln_ineq_con_3 [ -4.3644298963447897e-01 1.4999999999999999e-01 ] obj_fn gradient [ 1.3855136437818300e-01 0.0000000000000000e+00 ] nln_ineq_con_1 gradient [ 0.0000000000000000e+00 1.4999999999999999e-01 ] nln_ineq_con_2 gradient [ 0.0000000000000000e+00 -1.9485571585149869e-01 ] nln_ineq_con_3 gradient ------------------------------------------ Restart record 2 (evaluation id 2): ------------------------------------------ Parameters: 2.1640000000000001e+00 intake_dia 1.7169994018008317e+00 flatness Active response data: Active set vector = { 3 3 3 3 } -2.4869127192988878e+00 obj_fn 6.9256958799989843e-01 nln_ineq_con_1 -3.4245008972987528e-01 nln_ineq_con_2 8.7142207937157910e-03 nln_ineq_con_3 [ -4.3644298963447897e-01 1.4999999999999999e-01 ] obj_fn gradient [ 2.9814239699997572e+01 0.0000000000000000e+00 ] nln_ineq_con_1 gradient [ 0.0000000000000000e+00 1.4999999999999999e-01 ] nln_ineq_con_2 gradient [ 0.0000000000000000e+00 -1.6998301774282701e-01 ] nln_ineq_con_3 gradient ...<snip>... Restart file processing completed: 11 evaluations retrieved.
A Dakota restart file can be converted to a neutral file format using a command like the following:
dakota_restart_util to_neutral dakota.rst dakota.neu
which results in a report similar to the following:
Writing neutral file dakota.neu Restart file processing completed: 11 evaluations retrieved.
Similarly, a neutral file can be returned to binary format using a command like the following:
dakota_restart_util from_neutral dakota.neu dakota.rst
which results in a report similar to the following:
Reading neutral file dakota.neu Writing new restart file dakota.rst Neutral file processing completed: 11 evaluations retrieved.
The contents of the generated neutral file are similar to the following (from the first two records for the Cylinder example in[4]).
6 7 2 1.8000000000000000e+00 intake_dia 1.0000000000000000e+00 flatness 0 0 0 0 NULL 4 2 1 0 3 3 3 3 1 2 obj_fn nln_ineq_con_1 nln_ineq_con_2 nln_ineq_con_3 -2.4355973813420619e+00 -4.7428486677140930e-01 -4.5000000000000001e-01 1.3971143170299741e-01 -4.3644298963447897e-01 1.4999999999999999e-01 1.3855136437818300e-01 0.0000000000000000e+00 0.0000000000000000e+00 1.4999999999999999e-01 0.0000000000000000e+00 -1.9485571585149869e-01 1 6 7 2 2.1640000000000001e+00 intake_dia 1.7169994018008317e+00 flatness 0 0 0 0 NULL 4 2 1 0 3 3 3 3 1 2 obj_fn nln_ineq_con_1 nln_ineq_con_2 nln_ineq_con_3 -2.4869127192988878e+00 6.9256958799989843e-01 -3.4245008972987528e-01 8.7142207937157910e-03 -4.3644298963447897e-01 1.4999999999999999e-01 2.9814239699997572e+01 0.0000000000000000e+00 0.0000000000000000e+00 1.4999999999999999e-01 0.0000000000000000e+00 -1.6998301774282701e-01 2
This format is not intended for direct viewing (print
should be used for this purpose). Rather, the neutral file capability has been used in the past for managing portability of restart data across platforms (recent use of more portable binary formats has largely eliminated this need) or for advanced repair of restart records (in cases where the remove
command was insufficient).
Conversion of a binary restart file to a tabular format enables convenient import of this data into 3rd-party post-processing tools such as Matlab, TECplot, Excel, etc. This facility is nearly identical to the output activated by the tabular_data keyword in the Dakota input file specification, but with two important differences:
tabular_data
(i.e., any internal finite difference evaluations are included).An example command for converting a restart file to tabular format is:
dakota_restart_util to_tabular dakota.rst dakota.m
which results in a report similar to the following:
Writing tabular text file dakota.m Restart file processing completed: 10 evaluations tabulated.
The contents of the generated tabular file are similar to the following (from the example in the Restart section of[4]). Note that while evaluations resulting from numerical derivative offsets would be reported (as described above), derivatives returned as part of the evaluations are not reported (since they do not readily fit within a compact tabular format):
%eval_id interface x1 x2 obj_fn nln_ineq_con_1 nln_ineq_con_2 1 NO_ID 0.9 1.1 0.0002 0.26 0.76 2 NO_ID 0.90009 1.1 0.0001996404857 0.2601620081 0.759955 3 NO_ID 0.89991 1.1 0.0002003604863 0.2598380081 0.760045 4 NO_ID 0.9 1.10011 0.0002004407265 0.259945 0.7602420121 5 NO_ID 0.9 1.09989 0.0001995607255 0.260055 0.7597580121 6 NO_ID 0.58256179 0.4772224441 0.1050555937 0.1007670171 -0.06353963386 7 NO_ID 0.5826200462 0.4772224441 0.1050386469 0.1008348962 -0.06356876195 8 NO_ID 0.5825035339 0.4772224441 0.1050725476 0.1006991449 -0.06351050577 9 NO_ID 0.58256179 0.4772701663 0.1050283245 0.100743156 -0.06349408333 10 NO_ID 0.58256179 0.4771747219 0.1050828704 0.1007908783 -0.06358517983 ...
Controlling tabular format: The command-line options --freeform
and --custom_annotated
give control of headers in the resulting tabular file. Freeform will generate a tabular file with no leading row nor columns (variable and response values only). Custom annotated format accepts any or all of the options:
header:
include %-commented header row with labels eval_id:
include leading column with evaluation ID interface_id:
include leading column with interface IDFor example, to recover Dakota 6.0 tabular format, which contained a header row, leading column with evaluation ID, but no interface ID:
dakota_restart_util to_tabular dakota.rst dakota.m --custom_annotated header eval_id
Resulting in
%eval_id x1 x2 obj_fn nln_ineq_con_1 nln_ineq_con_2 1 0.9 1.1 0.0002 0.26 0.76 2 0.90009 1.1 0.0001996404857 0.2601620081 0.759955 3 0.89991 1.1 0.0002003604863 0.2598380081 0.760045 ...
Finally, --output_precision integer
will generate tabular output with the specified integer digits of precision.
In some instances, it is useful to combine restart files into a single master function evaluation database. For example, when constructing a data fit surrogate model, data from previous studies can be pulled in and reused to create a combined data set for the surrogate fit. An example command for concatenating multiple restart files is:
dakota_restart_util cat dakota.rst.1 dakota.rst.2 dakota.rst.3 dakota.rst.all
which results in a report similar to the following:
Writing new restart file dakota.rst.all dakota.rst.1 processing completed: 10 evaluations retrieved. dakota.rst.2 processing completed: 110 evaluations retrieved. dakota.rst.3 processing completed: 65 evaluations retrieved.
The dakota.rst.all
database now contains 185 evaluations and can be read in for use in a subsequent Dakota study using the -read_restart
option to the dakota
executable.
On occasion, a simulation or computer system failure may cause a corruption of the Dakota restart file. For example, a simulation crash may result in failure of a post-processor to retrieve meaningful data. If 0's (or other erroneous data) are returned from the user's analysis_driver
, then this bad data will get recorded in the restart file. If there is a clear demarcation of where corruption initiated (typical in a process with feedback, such as gradient-based optimization), then use of the -stop_restart
option for the dakota
executable can be effective in continuing the study from the point immediately prior to the introduction of bad data. If, however, there are interspersed corruptions throughout the restart database (typical in a process without feedback, such as sampling), then the remove
and remove_ids
options of dakota_restart_util
can be useful.
An example of the command syntax for the remove
option is:
dakota_restart_util remove 2.e-04 dakota.rst dakota.rst.repaired
which results in a report similar to the following:
Writing new restart file dakota.rst.repaired Restart repair completed: 65 evaluations retrieved, 2 removed, 63 saved.
where any evaluations in dakota.rst
having an active response function value that matches 2.e-04
within machine precision are discarded when creating dakota.rst.repaired
.
An example of the command syntax for the remove_ids
option is:
dakota_restart_util remove_ids 12 15 23 44 57 dakota.rst dakota.rst.repaired
which results in a report similar to the following:
Writing new restart file dakota.rst.repaired Restart repair completed: 65 evaluations retrieved, 5 removed, 60 saved.
where evaluation ids 12
, 15
, 23
, 44
, and 57
have been discarded when creating dakota.rst.repaired
. An important detail is that, unlike the -stop_restart
option which operates on restart record numbers, the remove_ids
option operates on evaluation ids. Thus, removal is not necessarily based on the order of appearance in the restart file. This distinction is important when removing restart records for a run that contained either asynchronous or duplicate evaluations, since the restart insertion order and evaluation ids may not correspond in these cases (asynchronous evaluations have ids assigned in the order of job creation but are inserted in the restart file in the order of job completion, and duplicate evaluations are not recorded which introduces offsets between evaluation id and record number). This can also be important if removing records from a concatenated restart file, since the same evaluation id could appear more than once. In this case, all evaluation records with ids matching the remove_ids
list will be removed.
If neither of these removal options is sufficient to handle a particular restart repair need, then the fallback position is to resort to direct editing of a neutral file to perform the necessary modifications. <!-----------------------------------------—>