Tideda for Windows Version 4.1 Release Notes.
Tideda Version: 4.1.10
Last Modified: 28 September 2005
This document describes enhancements and bug fixes since the publication of the Tideda for Windows Reference Manual (NIWA Technical Report 88, S M Thompson, August 2000). It includes features included in previous versions.
For installation instructions see td32_installation.doc
For later changes and bug fixes, see the current Readme.txt.
Contents:
Technical
Note: Pasting Graphs
Tideda Version 4 (Major Release):
New Features since the Tideda
Reference manual:
Labelling Gaugings on Rating Plots
Modified Delete option in Pledit
Gauging Data (Important Feature)
Data Import Wizard in script files
Run Time Entry of Site Details
Microsoft .NET framework
Tideda Version 4.1.01 or later requires the Microsoft .NET framework version 1.1 to be installed on the host PC. If not already present this can be installed using the DOTNETFX.EXE supplied on the Tideda CD. The Microsoft database engine may also need to be installed using MDAC_TYP.EXE. The Microsoft Visual C++ runtime libraries (mfc71.dll, msvcr71.dll) are also provided on the CD (these can be copied into your winnt\system32 directory if not already present).
You will need to install the Microsoft .NET framework if you get a message such as:
"The application failed to initialise..."
when starting Tideda. This usually indicates that the .NET framework is not installed on your machine.
Network Install
Added "Network Install" to installation "Setup Types". This option is the same as "Typical", but it copies the Fortran and Tideda shared Run-Time libraries (Dforrt.dll, Td8_Tilib.dll, TdAttribInterface.dll, TdAttrib.dll) to the Tideda program directory rather than to the Windows System directory. This allows the installation to be done directly to a network folder where it can be accessed by other users. The users will need to set their working directory and default attribute file using the Tideda "Preferences".
Because the attribute file manager in this version of Tideda uses Microsoft .NET code you will need to use the “.NET Framework Configuration” utility (Control panel à Administrative Tools) to assign “Full Trust” to TdAttribInterface.dll and TdAttrib.dll for each workstation.
*NOTE: If the Tideda software is installed on a network drive it can only be used by Windows XP workstations. There is a problem with starting this version of Tideda (version 4.1.01 or higher) from a network drive when the workstation is using the Windows 2000 Operating System. This hasn’t been tested with other Windows OS (98,NT).
Pasting Graphs
Technical note: When copying Tideda graphics to Microsoft Word via the Windows Clipboard the result varies depending on the version of MS-Word used. The paste operation works correctly when using MS Office 97 (all releases), MS Office 2000 (service pack 2 only) and MS-Office 2003. Other versions of MS-Office Word may not display the graph correctly, but later versions should be OK. The operating system version does not seem to affect the result (Windows 98,NT4, 2000, XP are all OK with the versions of MS-Word mentioned above). If the graph does not paste correctly in Word 2000 (particularly the axis labels), you will need to upgrade to service pack 2.
It is recommended that you use “Paste Special à Picture (Enhanced Metafile).
Note: Text in graphics pasted to Word 97 by Tideda for Windows versions prior to 3.2.05 may not display correctly in any version of Word 2000. These graphs must be re-created and pasted with a later version of Tideda.
New Features in this version
Extra Gauging Items:
Extra gauging items have been added to change standard gaugings from 15 items to 18 items.
The 18 items are:
Items 1 to 14 are the same as for 15 item gaugings.
Item 15 is Gauging Uncertainty (percent +/-).
Item 16 is Comment Flag. (0=no comment, 1=comment in comment file)
Item 17 is Gauging Agency.
Item 18 is Gauging number.
If you are changing to the 18 item form there is a command in the Tideda "Extras" menu to convert all gaugings in the current input file to 18 items. This also writes the existing 15 item gaugings to a Tideda list file as a backup. The name of this list file is “<name>_15Item_gaugings.ed”, where <name> is the name of the current Tideda input file.
The NIWA TdGauge software includes the calculated uncertainty when sending gauging results to Tideda.
The “Gauging Agency” is a hard coded list
of
The “Gauging Uncertainty” (item 15 of an 18 item gauging) is used when plotting error bars in rating plots with gaugings enabled. The” Gauging Agency” is shown when labelling gaugings on a rating plot and can also be used to “filter” the gaugings to only include gaugings by the required agency.
The “Gauging Agency” filter can also be used in the rating quality processes Qbedplot, Qreport & Qstat and when plotting gaugings on a time-series graph.
Tideda Parameter File:
The way the parameter file "Tideda.par" is used has been changed so it is only accessed when Tideda first starts and when it exits. While operating, Tideda uses a temporary copy of the parameter file. This is done to avoid parameter file access conflicts when running multiple copies of Tideda concurrently on the same PC. The temporary parameter file will be named "Tidedn.par" where 'n' is a digit 0-9.
GRDC Data Export Format:
The "GRDC yyyy-mm-dd" format has been added to Data Export for use when sending discharge data to WMO Global Data Runoff Centre. This date format also enables other data formatting features to make the output compatible with GRDC input.
Data Import Wizard:
An "Insert Zero" field has been added to the Import Wizard for Incremental data (e.g. event rainfall data). This option specifies the maximum time (in minutes) between readings before a "zero" is inserted to establish the start of a new accumulation. For example: With "Insert zero" set to 15 minutes, if one rainfall tip is recorded at midday and the next is recorded at 1pm, then at value of zero will be filed at 12:45pm to indicate that the 1pm tip is accumulated since 12:45, rather than since midday.
Export entire file as XML:
The “XML Export” process now has an option to “EXPORT entire File”. This dumps all data from the current input file into one or more XML files. This is used to convert the Tideda data file into a self describing transfer format. If the output XML is not in a “single XML file” it is split into one file for each site in the input file. The file is named with the Tideda “List” file name appended with the site number and the extension “.xml” (e.g. “c:\temp\xfer_65104.xml”)
Rainfall Intensity Graph:
A "Rainfall Intensity/Duration/Frequency" graph (PLIDF) has been added to the Graph menu (Graph à Special à Rainfall Intensity).
This plots the rainfall intensity (or depth) against rainfall duration for a pre-defined range of event durations from 2 minutes to 4 days. There is also an option to display a set of user-supplied frequency (or return period) curves.
Both vertical (intensity) and horizontal (duration) axes can display either linear
or logarithmic scales.
The information is also displayed in tabular form in the Tideda text window (View-->Text).
This process may be run on any period of data, from a single rainfall event, to an entire rainfall record of many years. The selected period is processed once for each of the pre-defined durations.
When "Auto" range is selected for the duration axis the durations included are: 6 minute, 10 minute, 20 minute, 30 minute, 1 hour, 2 hour, 6 hour, 12 hour, 1 day, 2 day, 3 day and 4 day. With "Auto" turned off you can select a range to include more pre-defined durations (2 minute, 7 day, 30 day, 1 year and 10 year), or to reduce the range of durations.
If the "Frequency Curves" option is specified the curves are read from a file supplied by the user. The format of this file is based on the NIWA HIRDS (High Intensity Rainfall Data System) output. This is a table of ARI (Average Recurrence Interval) values for a range of durations.
Tideda can read the HIRDS output files, but it can also use tables created from other sources as long as the format is compatible with the table reader. The reader looks for a header line beginning with "ARI" and reads the durations from this line. It then reads the ARI (years) and the rainfall depth values (mm) from the following lines. It stops when it reaches line that doesn't begin with a number. The values can be either space or comma delimited.
Example of Frequency curve file:
Long Ridge
rainfall station
Duration
ARI 10m
20m 30m 60m
2h 6h 12h
24h 48h 72h
2 11.9
20.0 27.2 45.8
73.9 157.8 254.7
411.0 542.6 638.4
5 15.5
25.7 34.2 53.0
87.2 185.3 301.9
515.3 682.6 799.5
10 18.2
28.8 37.6 59.6
96.7 208.5 338.4
549.4 734.3 870.1
20 21.5
33.2 42.8 66.1
107.6 232.8 378.7
616.3 827.7 983.5
50 26.9
40.1 50.8 75.9
123.9 269.3 439.6
717.4 969.7 1156.6
100 31.9
46.4 57.9 84.4
138.0 301.1 492.7
806.0 1094.8 1309.6
150 35.3
50.6 62.6 89.8
147.1 321.7 527.1
863.5 1176.3 1409.5
This table
can have up to 20 durations and up to 20 recurrence periods.
Tideda Attribute File:
A new Attribute database format that supports up to 255 items has been implemented in Tideda.
The new attribute format uses a relational database (e.g. Microsoft Access) and is not compatible with the earlier proprietary Tideda attribute file format that only supported 15 items (e.g. sites.att).
A template MS-Access file (Tideda_Attrib_Template.mdb) is copied into your Tideda working directory by the Tideda installation procedure
You can use either this new format or the old “.ATT” attribute file format by specifying the attribute file in the Tideda “Preferences” screen.
There is no urgent need to start using the new format unless you need to want to have attributes with more than 15 items (so it would be useful for 18 item gaugings). If you keep using the “Old style attribute file” there should be no major change to the editing and use of site attribute information.
To add a new site to the new attribute database use Edit àEdit Attributes to start the attribute editor.
In the "Edit Site" box set the site number and data kind (or select them from your data file). Enter the site name and select an existing Attribute Type from the list. You can then enter more site information using the "Site Details" button.
If the required “Attrib Type” is already defined this is all that is required to add the new site.
If a suitable type doesn't exist you will need to add one using the "Edit Types" button. A "type" defines the measurements that are assigned to each Tideda item (unrated or rated). You may also need to add new "Measurements" to complete a "Type" definition.
Each “Item” that is used by the “Type” is selected from the list of “Measurements”.
“Measurements” describe the quantities that are measured and the way they are displayed.
The "Display Mask" can be used by the List and Export processes to select the items to be included.
This version also changes the LIST process to list all items across a scrollable window rather than wrapping long lines. This allows each column to be read below the correct item heading. When printed this listing is printed across multiple pages.
Event Analysis:
The seasonal filtering options for the Event analysis processes have been modified. Seasonal filtering is now explicitly enabled by the "Seasonal filter" option. The start and finish months are now selected from a list of month names rather than by typing a number from 1 to 12. When the seasonal filter is turned off the month fields show the start month based on the start time of the data.
A bug has been fixed in the "Fred" (frequency distribution) event analysis process. When the "normal" distribution was used the "standard deviation" value displayed was incorrect. This value should be the "L2" value multiplied by the square root of PI, but we were multiplying the value by PI. An extra line has been added to clarify that the "location" and "scale" are L-moments estimates. For the normal distribution these values are now referred to as "location" and "scale" to reinforce that they relate to the distribution function rather than the sample data.
The "Fred" frequency distribution plot has been changed to display the distribution flow values in data units when the distribution is fitted to logged data. In this case the uncertainties are displayed as "factorial standard error" values rather than "1.96 standard deviations". The upper and lower bounds can be calculated using the factorial standard error. The distribution moments are still displayed in logged units.
Plotting Gaugings:
A new process (Graph à Gaugings) has been added to the Graph menu. This allows gaugings to be explicitly plotted on Stage vs Flow axes without plotting the rating curves as with (Graph à Ratings).
User Prompting in PSIM:
New “Prompt” and “Xprompt” commands have been added to the PSIM source code command set. This allows the PSIM source code to include information that will be displayed in the prompt field in the option screen when the PSIM program is selected. The form of these commands is “$$$ prompt: <text>” for text to be shown in the main PSIM option screen or “$$$ xprompt: <text>” for text to be shown in the secondary option screen when the XGET command is used. As these commands are implemented as comment lines in the PSIM source code there is no change to the run-time behaviour of PSIM, the information is only used by the interactive PSIM option screens.
Example (toy.sim):
$$$ prompt: Toy.sim counts the number of gaugings.
$$$ prompt: Make sure you select gauging data.
$$$ xprompt: We do an XGET just for the hell of it
$$$ xprompt: and ignore the value.
xlock
init count=0
get a
xget b
count=count+1
endl
print count
Tideda
Version 4 (Previous Major Release):
Summary
Tideda for Windows Version 4 (or “Tideda
4”) was a major release of Tideda that introduced significant changes to the
database and to the output format. The
database changes were required to extend the
Tideda 4 is able to read from and write to earlier version Tideda files – including “extended directory” data files. The Tideda 4 database format is completely different from earlier versions – so files created in version 4 can only be read by Tideda 4 and by software using the new Tideda 4 database access library - perversely named TD8_Tilib.dll (because it uses 8-byte integer values).
When creating a new data file, you have the option to create an “old version file” if it may be used by an old version of Tideda or other software that can’t read the new file format. This option will obviously not work with data past the old 2008 Tideda date limit.
Details
Dates:
The allowable date range for the new Tideda database format is 1st January 109 AD to 31st December 9999. This range is imposed by the user interface date format – the internal database time range is potentially much greater than this.
The old Tideda date input format (“yyymmdd”) is fully supported by Tideda 4 , so existing Tideda script files will run correctly in either version of Tideda. This includes the short form dates such as “102” for 1-Jan-2002.
The new date input format is similar to the old format, but uses the full 4 digit year “yyyymmdd” (e.g. 20020701). Unless otherwise specified, the calendar format of dd-mmm-yyyy is also acceptable (e.g. 1-Jul-2002). The short form in the new format is yyyy, so 2002 is 1-Jan-2002.
The Tideda date format for output is “yyyymmdd” and the Calendar date format is “dd-mmm-yyyy”. This option in the Tideda Preferences sets the date format for all text output.
Site Numbers and Items:
The maximum Tideda site number was increased from 8,388,607 to 999,999,999,999 (full 12 digits).
The maximum number of data items for each data element was increased from 15 to 255.
The allowable range of data values remains at +/- 2,147,000,000.
Compatibility and Conversion:
Tideda 4 is able to read from, and write to, earlier version Tideda data and attribute files.
A version 4 file cannot be read by earlier versions of Tideda.
Converting an earlier version file to the new Tideda 4 format is done by “Releasing” the file without selecting the “Create Old version files” option.
A version 4 file can be converted back to the old format in the same way – provided the file doesn’t contain data that is incompatible with the old format (date range, site number or number of items). You might want to do this to send data to someone that doesn’t have Tideda 4 (which should now been uncommon).
The “Create Old version files” option in the Tideda Preferences dialogue sets the default file version. When checked this option causes Tideda to create “Old” version files unless told otherwise. When unchecked this option causes Tideda to create new “ version 4” files unless told otherwise.
To force the creation of an old version file in a Tideda script file you must use the /C switch on the Release or Create command (e.g. Create 256 /C to create an old version file with room for 256 batches of data).
To force the creation of a new version 4 file in a Tideda script file you must use the /4 switch on the Release or Create command (e.g. Create 256 /4 to create a version 4 file with room for 256 batches of data).
The new version of Tideda no longer uses the file “Timouse.dat” to store it’s option settings. This version now uses a file named “Tideda.par”. These two files are not interchangeable.
User programs written for the Tideda PSIM process should run normally in Tideda version 4 – except that the date returned by the “TIME” command is in the Tideda “yyyymmdd” date format rather than the old “yymmdd”. You will need to check your PSIM program sources to see if this might affect the operation of your program.
New features since the Tideda Reference Manual:
XML Data Exchange:
Tideda can export Tideda data as an XML document and import data from Tideda compatible XML documents. These XML document files can be used to exchange data between users, or to export data in a form that can be displayed and browsed in a web browser. The Tideda XML document is a self describing transfer format that includes the site attribute information, which can be used to update the Tideda attribute record for the site.
The Tideda XML Document Type Declaration (DTD) can be included in the exported XML file if required – but Tideda can import the Tideda XML data without it.
By exporting Series data, Gaugings and Ratings for a site the complete site can be exchanged. Remember to “append” to the list file when exporting for more than one site or kind of data.
To try out this facility just XML Export some data to a file and open the XML file with a web brower (e.g. MS Internet Explorer), or XML Import it back into a new Tideda file.
See sample XML file: Tideda4_Release_Notes_files\LIST.XML
There is also an option to “EXPORT entire File”. This dumps all data from the current input file into one or more XML files. This is used to convert the Tideda data file into a self describing transfer format. If the output XML is not in a “Single XML file” it is split into one file for each site in the input file. The file is named with the Tideda “List” file name appended with the site number and the extension “.xml” (e.g. “c:\temp\xfer_65104.xml”)
Comments:
The Tideda Comment database interface has been completely revised. This documentation replaces any earlier description of Tideda comment database functionality.
All comment facilities are grouped under the "Comments" menu. This includes two new processes: Comment Export and Comment Import.
Comment Export is similar to the Comment "Table" process - but it writes the comments to the Tideda list file in the Tideda "text file" comment format with the "@@ <site> <date> <time>" header. Export is used to extract comments for particular sites from the comment database. This can be run from a script file using the "CExport" command.
Comment Import reads comments in the Tideda "text file" comment format and writes them to the current comment database (either text file or ODBC database). This should be used for comment transfer and updating. If a comment already exists in the current database for the same site, date and time both comments will be displayed and you are given the option to "Add" the new comment (leaving the existing one in the database), "Replace" the existing comment with the new one or "Skip" the new comment and retain the existing one.
Comment Database:
Tideda can work with comments filed in the existing text based comment files or from an ODBC data source conforming to the definition given below. Use the MS-Access database “Tideda_Comments_Template.mdb” as a template. This file is supplied in the Tideda installation.
The format of the ODBC comment database is:
In a table named “Data” there must be the following fields in the specified order:
ID: Long Integer. - Arbitrary record ID number. Use as the Primary key.
Site: Long Integer - The Tideda Site Number (maximum is approximately 2147000000).
Date: Long Integer - The date of the comment in Tideda format (yyyymmdd)
Time: Long Integer - The time of the comment in Tideda format (hhmmss)
Comment: Memo field - The text of the comment.
Author: Text (50) - The name of the person or organisation filing the comment.
Type: Text (20) - The comment type (selected from the “Comment_Type” table).
DataType: Text (12) - The Tideda data type for which the comment is relevant. This is selected from a pick list containing the following values: “All”, “Series”, “Gauging”, “Rating”, “<undefined>”.
SiteType: Text (20) – One of the following types: "Flow-Level", "Rainfall", "Misc", "Water Quality", "Ground Water", "Climate", "Sediment"
The “Comment_Type” table contains a single column named “comment_type”. This is a list of the valid comment types as defined by the user or organisation. A limited set of types is suggested to avoid unnecessary complexity. For example:
“Site” for comments about site location, physical features and recording authority.
“Equipment” for details about the site equipment and structures.
“Data” for comments about missing data, data quality and gaugings.
“Rating” for comments about ratings or rating changes.
“Miscellaneous” for any that don’t fit into one of the other categories.
Because records in the “Data” table can be added or modified by Tideda (using the comment editor or from the graphical data editor) it should only contain the fields described above. Your database can contain other tables if you require other information to be stored in the database.
This database may be defined in your ODBC data sources as “Tideda_Comments” (under the Windows Control Panel), or Tideda can access the database file directly. See the section on database connection below for details.
For a sample Microsoft Access database see “Tideda_Comments_Template.mdb” in the example files in the Tideda working directory (if installed). This contains the data definition and some comments for site 65104. You can rename (or copy) this file and delete the existing data records to use this as your Tideda comments database.
The older Tideda ODBC comment database format (described in the Tideda for Windows version 2.3 release notes, August 1997) is no longer supported. This was the one in which the Comment memo field is replaced by 14 character fields named Comment1 to Comment14.
Existing text based comment files can have some of the added functionality of the ODBC database by adding a “comment type” table to the start of the text file and adding header lines to the comments to assign type, data type and author fields. See “Tideda_Comment_Text.cmt” in the Tideda example files in the Tideda working directory (if installed). The comment type table has the format:
Typetable:
Site
Data
Equipment
Miscellaneous
Rating
endtable
If you want to convert existing comments from the Tideda text based comment format a conversion utility is available on request.
Comment Database Connection:
The Tideda comment database is selected using the “Comments” page in the Tideda Preferences dialogue (File → Preferences). The three main options are: None, ODBC Database and Text File
Use “None” if you don’t want to use comments.
Use “Text file” to read comments from old text based Tideda comment files. Just type in the file name or use Browse to specify the comment file name. Tideda can read from, and write to, these files.
Use “ODBC database” to access comments in an ODBC database:
If you have previously configured the database as “Tideda_Comments” in an ODBC Data Source definition you can select this under “ODBC Data Source”.
If you want to specify the actual ODBC database file name in Tideda you can select it under “Specify database file”. You will also need to select the ODBC database driver to use (e.g. “Microsoft Access Driver (*.mdb)”) and any user name and password required to access the database (these are also required when setting up an ODBC data source in the Windows control panel).
ODBC Configuration:
The ODBC driver for the database is supplied with the database software (e.g. MS Access) and is installed on your system when the database software is installed - it is not a component of Tideda. These notes assume that you have already installed the database software correctly and entered comments into a comment database.
· Open the “ODBC Data Sources” administration utility in the MS-Windows Control Panel. This may be under “Administrative tools”.
· Choose the “Add” option, select the ODBC driver for your database type (e.g. Microsoft Access Driver) and press OK.
· The Data Source set up panel will appear. In the Data Source Name field enter the name “tideda_comments”.
· In the “Database” section choose “Select”.
· Make the connection to your database by selecting it from the appropriate directory then press OK.
Comment Editor:
The Tideda comment editor (Edit → Comments) allows interactive viewing, searching and editing of comments in the currently connected database (ODBC or Text file). This is independent of any data entry form defined in the database application – for example, the Tideda_Comments_Template.mdb supplied with Tideda has a simple MS-Access data entry form that could be used outside of Tideda.
The Tideda comment editor has filters to select records based on comment type and date range. It allows the user to display records for a selected site or for all sites in the comment database.
Comments in Tideda:
Outside of the comment editor, comments can be displayed in several ways:
Pcomment (Comments à Table) prints comments for the selected site and time range.
Plzoom (Graph → Graph and Zoom) shows comments as triangular markers on the bottom time axis. If there are comments in the selected time range the “Comments” option will appear in the zoom menu. If you select this option you can display the comment text in a window by clicking near to one of the triangles. There is an option in this window to copy the comment text to the Windows clipboard so it can be transferred to another application.
Plgraph (Graph à Graph over Time) shows comments as triangular markers on the bottom time axis.
Pday (Table → Calendar → Daily) indicates where comments are filed by displaying an exclamation mark beside the value.
Plpscan (Manage → Graph scan) has an option to show the time of comments with a triangular marker.
Graphs:
Graph Pscan:
There is a new process called Plpscan (Graph Pscan in the “Manage” menu). This process is based on the Plscan procedure developed by Roddy Henderson at NIWA. It produces a graph that shows a time line of the data coverage for a range of sites in the current Tideda input file. It is useful for data quality checking and for showing where data is available across a range of sites.
Plpscan implements several optional filters to control the content of the plot. The filters are independent and can be used in any combination. The available filters are:
Site Mask – this includes sites that match the site number mask, e.g. 142* will include all site numbers beginning with 142, *5*5* will include all site numbers containing two or more fives. This is the mask option in a script file (“mask <mask>” or “nmask” to include all site numbers).
Start Date – sets the start date of the plot and only includes sites that contain data at this time or later. This is the start option in a script file (“start <date>” or “nstart” to set the start date at run time). The actual time range of the data on file is still shown in the site description when a fixed start or end time is specified for the plot.
End Date – sets the finish date of the plot and only includes sites that contain data at this time or earlier. This is the end option in a script file (“end <date>” or “nend” to set the end date at run time).
Data Kind – includes only data of the specified Kind. This is the kind option in a script file (“kind <Tideda kind>” or “nkind” to include all kinds of data).
Other options for this process also control the content of the plot:
Show Site Number – displays the site number in the site description line. This is the site/nsite option in a script file.
Show Site Title – displays the site name (from the attribute file) in the site description line. This is the title/ntitle option in a script file.
Show Gaugings – marks the time of any gaugings for the sites on the time line. Each gauging is plotted as a small diamond mark centred on the time line for each site. This is the gaugings/ngaugings option in a script file.
Show Comments – marks the time of any comments for the sites on the time line. Each comment is plotted as a small triangle mark below the time line for each site. This is the comments/ncomments option in a script file.
The time line for each site shows the period covered by continuous data. Periods between batches of data are left blank. If a batch has a gap marker this is shown as a small vertical tick mark at the start of the batch. (Note: if there is a long space between batches, without a gap marker at the start of the new batch, this may indicate an unmarked period of missing data).
The thickness of the time line is set by the Data pen (dpen) option.
If there are more sites selected by the filters than can be drawn within the plot height this is indicated by “More…” in the bottom right corner of the plot.
Labelling Gaugings on Rating Plots:
There is an option to label the gaugings on a rating plot with the date and time of the gauging:
· The gauging label option is a simple on/off option with no other control over the style or positioning (other than by changing the plot size and ranges to include/exclude gaugings and position the gaugings on the plot).
· The Date format is controlled by the "Calendar" option (calendar format or Tideda format).
· The font size of the gauging dates is 3/4 the size of the font of the other text on the plot.
· The labels are in a single vertical column with a maximum of 100 date labels, or up to the height of the plot (which ever is smaller).
· In a Tideda script file this is the "date" or "ndate" option.
Copy Graph to File:
The Copy Graph to File command in the Edit menu copies the current graph window to an enhanced metafile on disk. These files can be viewed, imported into a document or transmitted to others. The equivalent command in a Tideda script file is ”PLFILE /F <name>”, where <name> is the name of the file to be created. If a path name is not included the file will be created in the Tideda working directory as specified in the Tideda Preferences dialogue. The enhanced metafiles always have the file name extension “.emf’. This facility can be used in a script file to create a series of graphs that can then be included in a document without copying and pasting one at a time.
Undo Graph:
An "Undo" option has been added to the Graph Menu. This command deletes the latest graph layer from the current plot.
Scatter Plot from two sites:
Tideda now allows the PLSCAT process (Graph à Scatter Plot) to plot data from a different site on each axis by activating the "2nd Site" option in the process option dialogue. Both sites must be in the current source file. This re-enables the "PLSCAT 2" command from the old DOS Tideda. The "PLSCAT 2" syntax is still supported in Tideda script files.
PLZOOM enhancements:
The PLZOOM process (Graph à Graph and Zoom) has been modified to allow panning in time and to "zoom out" one level at a time. It also actions a new zoom selection automatically when exiting the zoom mode rather than having to press the "plot zoom" menu button (which has been removed).
The "<Left" pan menu button moves the plot window back in time by 1/4 of the current time range and the "Right>" pan button moves the window forward in time. Panning is also available while in the "Zoom" mode by pressing the left and right arrow keys. In this mode the time window is moved by 1/10 of the current time range.
In the zoom mode the up and down arrow keys also allow vertical panning by 1/10 of the current vertical range.
Modified DELETE option in PLEDIT:
The DELETE option in the Point menu of the PLEDIT process (Graph --> Special --> Graph and Edit) now allows deletion of multiple points in a "point and click" manner, rather than requiring a data point to be selected first. The "undo" option restores all points deleted while the delete option was active.
Data Management
Multi-Item MERGE interface:
A new interface has been added to allow easier use of the MERGE process to assemble multiple source data items into a multi-item site (Move --> Merge --> Multi-Item). The source and destination details are assembled in the dialogue screen. This information is used to write a Tideda script file that performs the correct MERGE operations to give the required result. The first source site is used with the "MERGE 1" process to create the output site and set element times (required). The remaining source sites are the "MERGE 2" sites and are written to the selected item in the destination site. The new interface allows up to ten input items to be assembled. If more than this number of items are required the first ten can be assembled in this interface and any others can be added by editing the script file. The temporary script file created by this interface is "multimerge!.tsf" and is located in the Tideda working directory (see your Tideda "Preferences" for the name of the working directory).
Delete Site:
The "Delete Site" command has been added to the "Manage" menu. This command deletes all data for the specified site from the current input file (series data, ratings and gaugings). It deletes the entire time range - irrespective of the time range selected in the Data dialogue.
Update:
The “Update” process now uses a 32 kbyte batch allocation size when copying series data. This is to avoid excessive batches when updating data frequently. (see process RELEASE for information on free space allocation).
Copy All:
The COPY process now has an option to copy all data kinds for a site (series, gaugings and ratings) in the selected time range with a single command.
The Copy command in the Move menu has been split into three sub-commands:
Series data
Ratings
All kinds
The Copy All Kinds command in a script file is specified as ‘COPY A’. When using Copy All Kinds the rating that applies at the start time is the first rating copied rather the rating filed after the start time.
Gauging Data (Important feature!):
From Version 3.0.11 Tideda changed the way in which GAUGING data is copied.
Previously gaugings were written as a new series that would completely replace any existing gaugings for the site in the same time range. This could lead to the loss of all other gaugings if an early gauging and a recent gauging were written to a site in the same copy operation. Now gaugings will be 'merged' with any existing gaugings in the time range. This is a change to the basic operation of Tideda that better represents the fact that gaugings are not a continuous record.
This change applies to the Tideda processes COPY, INCLUDE and COMBINE. A message will be displayed by these processes when existing gaugings are detected on the destination file to alert the user to this.
To merge the new gaugings with the existing gaugings a temporary Tideda file is created in the Tideda working directory, which is specified in the Tideda ‘Preferences’. If you don't have write access to this directory the copy will fail.
This feature has been available for some time in TLIST by adding the MERGE keyword to the header line in the list file. Tideda now automatically adds this keyword when writing gauging data to a list file.
Moving Means:
New process "TMOVE" (Move ==> Moving Means) which transforms series data into moving means (or moving totals for incremental data). The output site is always single item.
The options for this process are:
Averaging Interval - e.g. 1 hour, 12 hour, 24 hour.
Destination file, Destination site - Should be different from the source data.
Destination kind - Normally Instantaneous, even for incremental input data
since the output values are not incremental values but
totals at a point in time.
Time to file - choose to file the output value at the Start of the interval,
the Middle of the interval or the End of the interval.
The "averaging window" moves through the source data so that a value is written every time the leading or trailing edge of the window matches a data time. There is no data compression, so the number of values generated could be up to twice the number of values in the input data. The output must be written to a new site or to a different destination file to avoid accidentally overwriting the input data. It is usually worth creating a new attribute for the output site with the appropriate descriptions.
Ratings
Rating Compression:
Added a facility to TLIST ("List to Tideda -> Fully Specified") that will handle ratings with more than 50 pairs of values. This compresses the rating to remove redundant pairs until there are no more than 50 pairs remaining. The initial compression is at 0.1% (i.e. a pair is removed if the flow calculated from any stage value with the pair removed is within 0.1% of the flow that is calculated with the pair included). The compression range is increased by 0.1% steps until the 50 pair limit is no longer exceeded. Normally the 0.1% compression is all that is needed. The maximum number of pairs this can process is currently set at 500.
Rating Table Format:
The rating table process PRATE ("Table -> Other -> Rating") has been changed to be more readable. It now appears as a single table without multipliers.
The presentation units divisor from the attribute file is applied (if specified) and the values are displayed as floating point numbers where
required. Up to six user defined alarm levels can be displayed at the bottom of the table.
Gap Ratings:
A facility to indicate that a previously rated series record is no longer rated. In Tideda the latest rating is applied infinitely into the future. If there is no longer a valid rating for the site you can now file an "unrated" or “gap” rating to indicate this. After this time Tideda will show a Gap where rated values are requested (or until a new valid rating is filed). An "unrated" rating is one where the first unrated value is the Tideda Gap value (2147000001). In practice the rating should be entered as three pairs of this value, e.g.
12345 RATING 1 1010201 120000 1010201 120000
2147000001 2147000001
2147000001 2147000001
2147000001 2147000001
Listing rated data:
There is the option to apply a rating when using the LIST process to display series data (List in the Tideda Table menu). To apply the rating you must select the correct item (Rated) from the Measurement list in the Data dialogue box. In a Tideda script file these are the ITEM and RATE (or NRate) options.
WARNING: Existing script files that list unrated values will not have the NRATE option specified. If the previous process had a rating applied the script will now list rated values for the item. You may need to add the NRATE option to these script files to avoid this.
Rating Quality Assurance:
Rating Quality processes have been added under the Extras menu. These three processes implement the facilities previously provided by the QRAT program for reporting on stage/flow rating curves and flow gaugings.
Bed Plot:
The Bed Plot process displays a print-plot of the “stage” deviation of the gauged flow from the rating and the percentage difference between gauged and rated flow for each gauging in the selected time range. This can be presented in sequential time order or sorted into decreasing flow order.
Standard Bed Plot for Site 65104
Stage Flow Date Deviation
mm l/s mm %
<-- Below rating Above rating -->
-500 mm 0 +500
Rating 941107 941109 |---|---|---|---|---|---|---|---|---|---|
1559 57926 950125 30 -5.4 | |* |
1027 19874 950216 2 -0.5 | * |
1278 34225 950428 6 -1.3 | * |
Rating 950506 950506 |-------------------|-------------------|
1423 44143 950607 -10 2.2 | * |
1299 30630 950815 20 -4.7 | |* |
Rating 950818 950818 |-------------------|-------------------|
1864 84872 950915 0 -0.1 | * |
1825 78920 951031 10 -1.5 | * |
1825 78890 951031 10 -1.5 | * |
Rating 951228 951229 |-------------------|-------------------|
1071 22176 960129 -8 1.7 | * |
1071 21775 960129 1 -0.1 | * |
1314 36833 960514 9 -1.8 | * |
1310 35509 960514 21 -4.5 | |* |
1320 37436 960626 7 -1.5 | * |
1319 37515 960626 5 -1.1 | * |
1311 37633 960801 -4 0.9 | * |
1313 37563 960801 -1 0.3 | * |
100% of Gaugings lie within +/- 8% of the ratings
Statistics:
The Statistics process shows gauging statistics for ratings in the selected time range. When the “sorted by flow” option is used statistics are shown for different flow segments of the rating range.
Standard Rating Statistics
**************************
Overall Statistics
==================
Number of gaugings 8 | Positive deviations 7
| Negative deviations 1
Number of runs 3
Maximum deviation -5.4 % on day 950125
Standard Deviation 2.3 %
Overall Bias (+/-) -1.59 % which is not significant at the 95% level.
Run statistics cannot be calculated.
Rule of thumb suggests that there may be some systematic error.
---------------------------------------------------------------------------
Individual Ratings
==================
Rating 941107 941109
-----------------------
Number of gaugings 3 | Positive deviations 3
| Negative deviations 0
Number of runs 1
Standard Deviation 2.2 %
Overall Bias (+/-) -2.39 % which is not significant at the 95% level.
Run statistics cannot be calculated.
---------------------------------------------------------------------------
Rating 950506 950506
-----------------------
There are only 2 gaugings defining this rating
Number of gaugings 2 | Positive deviations 1
| Negative deviations 1
Number of runs 2
Standard Deviation 3.4 %
Overall Bias (+/-) -1.23 % which is not significant at the 95% level.
Run statistics cannot be calculated.
---------------------------------------------------------------------------
Rating 950818 950818
-----------------------
Number of gaugings 3 | Positive deviations 2
| Negative deviations 0
Number of runs 1
Standard Deviation 0.7 %
Overall Bias (+/-) -1.02 % which is significant at the 95% level.
Run statistics cannot be calculated.
Standards Report:
The Standards Report produces the “Rating Standards Report” that shows how the flow gaugings match the ratings and how well the gaugings cover the rated flow range for both the selected time range and for the full time period on file. This process requires three statistics about the instantaneous flow record to be provided by the user: the long term minimum, maximum and 95% flow values. If the instantaneous flow record is available you can run the Summary process in the Tideda Statistics menu on the flow for the full period. This automatically sets these three values in the Standards Report option dialogue. If you do not have the flow record available you will need to obtain these values from somewhere else or make an educated guess.
The standards report is presented as a table in the Tideda text window and as a bar graph in the Tideda graphics window. To combine the text output and the bar graph into a single page report you will need to copy the text into a word processor package (e.g. MS-Word) and paste the graph in place of the distribution table. This can then be formatted to meet your requirements.
Rating Standards Report
Site 65104 Period 950101 to 960801 Compiled on 27-MAR-2001
Flow gaugings (for report period) conformance to standard:
Rating period No. of Gaugings No. of Gaugings
< 8% deviation
941107 - 950506 6 6
950506 - 950818 2 2
950818 - 951228 3 3
951228 - to date 8 8
Gauging coverage standard for 95% (of time) flow range:
The 95% estimated flow range for this station is 7793 to 123800 l/s
For this range 85.9% of gaugings lie within +/- 8% of the rated flow
Standard deviation is 6.7%
95% of the range is adequately defined by gaugings
Stage/discharge rating curve statistics (covering the full period and
full flow range of record):
Number of gaugings with positive deviations 293
Number of gaugings with negative deviations 298
Number of runs 291
84.9% of gaugings over the full range lie within +/- 8% of the ratings
Standard deviation is 6.8%
Distribution of Gaugings (full flow range):
No. of Gaugings <8% deviation
Flow Interval Total Period Report Period
7793 - 23261 69 3
23262 - 30994 76 1
30995 - 38728 77 7
38729 - 46462 46 1
46463 - 54196 43 0
54197 - 61930 36 1
61931 - 69663 21 0
69664 - 77397 20 0
77398 - 85131 14 3
85132 - 92865 13 0
92866 - 100599 10 0
100600 - 108332 9 0
108333 - 116066 6 0
116067 - 123800 4 0 < 95% flow
123801 - 293983 42 0
293984 - 464165 9 0
464166 - 634348 4 0
634349 - 804531 1 0
804532 - 974713 2 0
974714 - 1144896 0 0
PSIM
RUN Command:
Added the RUN command to PSIM to allow external commands to be run by the PSIM program.
Syntax: RUN <command_line>
For example:
RUN c:\working\alert "bad value in data"
Data Import
Data Import and ODBC Connection wizards:
The Data menu now includes Wizards to extract data from an ODBC data source or to import data from a variety of text files. The Export command can also be accessed from this menu.
ODBC
= Open Data Base Connectivity, a standard method that allows applications to
connect to a compliant database using SQL commands.
SQL =
Structured Query Language.
Data Import Wizard:
The Data Import wizard lets you interactively break a text file into data columns and define the contents of each column. These definitions can be saved in the template file and used again for data in the same format.
Start by choosing the column delimiters to break the text into columns, then “Freeze” the columns.
Select the columns to be imported (by clicking on the column headings) and define the contents of these columns. You must have Date and Time* (in separate columns) and at least one data column. If all the data is to go to one site this can be specified in the wizard dialogue, otherwise the site number column must be in the data with a valid Tideda Site number.
* If the values are daily readings the Time column can be omitted if the time of day is specified in the Time field of the wizard dialogue.
If the Import wizard can’t be configured to import your data you may be able to import it into a spreadsheet and massage it into a form that can be imported by Tideda (then save it as a CSV file and run the Tideda Import Wizard). The basic requirement for Tideda is that the date & time is increasing and the data values are numeric.
Data Import Wizard in script files
The Data Import Wizard can be used from a script file.
The syntax for this is:
import <text_file> <definition_name>
where <text_file> is the name of the text file to be imported and <definition_name> is the exact name of a definition previously saved in the Import Wizard definition file (tdImportWizard.txt). The comparison is not case sensitive.
The default location of the text file is the Tideda Working directory specified in the Tideda Preferences. The definition file is always in this directory.
If <definition_name> is not specified the Import Wizard dialogue will be displayed and can be used interactively to select the definition.
examples:
import mydata.csv My logger to site 99
import mydata.csv Flow data from NIWA
Restrictions:
The maximum length of the command line is 71 characters.
If the text file or definition is not found a message box will display the error. This will stall the script file until the error is acknowledged.
The site number is taken from the definition file (unless it is specified in the data). If you need to set the site number in the script file you will need to delete the fixed import site first, import the data to the site specified in the definition file, then copy this to the required site.
ODBC connection wizard:
The ODBC connection wizard allows you to connect to your data sources and create a query to extract data to a comma delimited text file. This file can then be imported into Tideda using the Data Import wizard.
You can save the SQL statements in a template file for later use. An ODBC driver for the Data Source must be installed on your system and the Data Source must have been previously defined in your system’s ODBC Data Source Administrator.
Choose ODBC Select in the Tideda Data menu. When the ODBC connection wizard is started it shows a list of the data sources on your system. Select the required source and click on Connect to establish the connection. If the database connection requires a user name or password you should type these in the fields provided.
After the connection has been established a list of the database tables will appear.
Double-clicking on a table name will display the names of columns in the table.
Double-clicking on a column name will display a preview of some of the values in this column
If a template SQL query has already been saved for this data you can use the Load SQL button to recall this query and (if necessary) edit it to match your requirements. Click on RUN SQL to run the query and save the data in the specified text file.
If you don’t have a template query defined for this data you can build one interactively by checking the select box and double-clicking on the required column names. This selects the columns that will be written to the output file. If the data is to be used by Tideda one of these will have to be a valid date (and probably a time). If a Tideda site number column is not available and the selection includes data from more than one site you should check the Auto site number box to generate a sequential site number in the output file.
If you want to apply a conditional selection you can check the where box and add the required conditions. For advanced selection queries you will need to have knowledge of SQL and the structure of the database that is being queried.
When a new query has been developed it can be added to the query template file with the Save SQL button. Give the query a meaningful description so it can easily be used again. The query template file is written to the Tideda Working Directory and is named odbcsql.txt.
Before the data retrieved by the wizard can be used in Tideda the relevant columns must written to a Tideda data file using the Data Import wizard.
Script Files
Relative time specification in Tideda script
files:
You can use this in a script file to specify a dynamic time range rather than explicitly entering start and finish times or using <D> or <U> arrows in the time entry fields. The syntax for this is:
#<key><delta_time>
where <key> is one of:
Now (current PC time)
End (end of data for the site)
Finish (previous finish time)
Start (previous start time)
(only the first letter is required, but it is best to use the full word for clarity)
<delta_time> is the relative time offset from the <key> date or time.
In a date field the <delta_time> is +/-yymmdd
In a time field the <delta_time> is +/-hhmmss
example:
plgraph n
65104
#now-30 (30 days before today)
120000 (at midday)
#now (today)
#now+100 (at 1 hour from now)
go
other dates:
#finish+10000 (1 year after the previous finish date)
#end-50600 (5 years & 6 months before the end of data date)
#start+600 (6 months after the previous start date)
other times:
#finish+10000 (1 hour after the previous finish time)
#end-50600 (5 hours & 6 minutes before the end of data time)
#start+600 (6 minutes after the previous start time)
This facility is intended to replace the Autoscript program used by NIWA.
Lerror command:
The LERROR command in Tideda script files is supported by Tideda for Windows. This signals Tideda to trap errors such as 'No data for site'. Tideda writes the error message and other information to a text file and exits with the error level set to 2. The syntax for this script file command is:
LERROR <filename>
where <filename> shouldn't specify a directory name - the file is created in the Tideda working directory (e.g. LERROR TDERR.TXT).
A batch file that runs a Tideda script file can check for "errorlevel 2" and take appropriate action.
When the LERROR command is used in a Tideda script file, or when a script file is run from the command line or batch file with the /Q option, Tideda will exit with errorlevel=2 rather than displaying an error dialogue that requires a user response. The error is logged in the error file specified in the LERROR command (in the Tideda Working directory). This file defaults to ERRLOG.TXT if no name is given.
This allows a batch file to run a Tideda script file and to trap any error returned from Tideda without stalling the procedure, for example:
cd \working
"c:\program files\niwa\tideda\td32.exe" @hourly.tsf/Q
if errorlevel 2 goto tiderr
echo Tideda script file OK
goto end
:tiderr
echo Tideda script file terminated by error
type errlog.txt
:end
Run-time entry of site details in Script files:
There is a new "DATA" command that can be used in a Tideda script file to display the Tideda Data dialogue to get site and time information from the user at run time. This command is placed before a process for which you wish to select the site or date when the script file is run. The following Tideda process should then use <D> to accept the given site and times or <R> to use the given site but override the times.
e.g.
DATA
COPY S
<D>
GO
or
DATA
COPY S
<R>
990101
0
1010101
0
GO
Pen Selection in Script Files
When running a Tideda script file the data pen option in graphs is:
DPEN <pen> <colour>
where <pen> is the Tideda pen number (in the range 1 to 99)
<colour> is the Tideda pen colour (in the range 1 to 64)
The pen number is translated to a line width in the range 1 to 10 by the calculation (Width=Pen/10 +1), (e.g. To specify a line width of 3, specify a pen number of 21). Tideda does not currently support broken line types.
The pen colour should be selected from the set of eighteen colours defined in Tideda. The colour numbers for these colours are:
|
3 - Black |
31 - Pale Green |
|
4 - Blue |
36 - Dark Grey |
|
13 - Navy Blue |
37- Light Grey |
|
14 - Sky Blue |
39 - Magenta |
|
18 - Cyan |
42 - Orchid |
|
19 - Firebrick |
47 - Red |
|
23 - Green |
56 - Turquoise |
|
25 - Olive Green |
62 - White |
|
26 - |
63 - Yellow |
For example, the Plgraph option ‘dpen 21,63’ sets the data pen to be yellow (pen 63) with a line width of 3 (=21/10 +1).
Colour numbers not in this table will be mapped to black.
The Axis pen option (APEN) has the same format as the DPEN option.
The background pen (BPEN) only specifies the pen colour (e.g. “BPEN 62” sets a white background).
Miscellaneous
Print File:
The old DOS Tideda Print File (PFILE) is emulated with the Copy to file command in the Tideda File menu. The command in a script file is "PFILE /F".
This copies the text in the current Tideda file window to the named file in the Tideda working directory.
Use "PFILE <name>" to set the print file name (maximum of 31 characters).
Use "PFILE /C" to clear the current window.
In version 4 Tideda the command can take the form “PFILE /F <name>” to send the text to the named file. In this case the file name can be longer than 31 characters and can include a path name.
The orientation of the text printout can be specified in a script file with the commands ‘PFILE /PORT’ or ‘PFILE /LAND’ for portrait or landscape.