Reliability is one of the most important requirements towards the control system. TRACE MODE® 6 has been designed to provide maximum control system reliability. It has a fail-proof project design system and it has built-in redundancy for all control system levels – from a single sensor, to the enterprise scale servers.
Redundancy features for the most components of the control system are provided automatically and no additional programming is required.
In the TRACE MODE IDE the redundant PC or PLC nodes may be created utterly easy – at one click of the mouse. Moreover, for the first time ever the control systems with dual and triple redundancy may be built with the same ease.
Let us consider the TRACE MODE features for the control systems reliability assurance more in details.
Sensors Redundancy and Diagnostics
The Micro TRACE MODE® reliability assurance system (for controllers) allows to control the quality of signals acquired from sensors and to provide their redundancy. In case of the break in communication with the sensors equipped with digital interfaces, the hardware invalidation will be stated for all signals received from them. Flags of hardware and software invalidation are transmitted in the TRACE MODE channels along with the value measured, as one of the channel attributes, and they can be used in algorithms. Developer can flexibly tune the sensor redundancy algorithms, using all the power of the 5 IEC 61131-3 standard programming languages.
There are no limitations whatsoever for redundancy of sensors or for its groups (for example, I/O cards). The dual and triple redundancy systems may be built easily. Therefore, TRACE MODE may be used to build control systems that monitor the sensor signal quality in real time and provide redundancy features that rise the overall system reliability.
Redundancy and Reliability for Controllers
TRACE MODE ensures hot redundancy for industrial controllers programmed in Micro TRACE MODE. This type of redundancy is used, as a rule, to assure reliability of control systems at hazardous processes.  Algorithms of controller redundancy can be flexibly adjusted by user and corrected in compliance with the requirements of a particular control system.
By default, the following hot redundancy technologies are implemented in TRACE MODE:
- channels database autobuilding for standby SOFTLOGIC controller;
- controller redundancy: switching-over of data flows to standby SOFTLOGIC controller in real time, in case of failure of the master;
- synchronization of real time data between the main and standby SOFTLOGIC controller.
Dual or triple redundancy provision for controllers does not exclude duplication of sensors. The sensors redundancy may be provided for every signal. The user himself can determine, whether each of dual redundant controllers shall get data of the given technological parameter from its sensor, or one sensor will be used as a source of information for both the controllers.
The TRACE MODE 6 reliability assurance system also includes the Watch Dog timer support, which helps to reboot controllers and industrial PCs automatically in case of the system halt.
The operation of all controllers in the control system is independent. The I-NET TRACE MODE network protocols ensures reliability and fail-safety of the control system. Even if a controller or server fails, this would not harm to the whole system – there will be no network overload due to continuing queries addressed to the failed device.
In case if other PLC programming system is used instead of Micro TRACE MODE, the control system reliability is assured in the following way: the hardware validation flag would be generated at the HMI PC, indicating availability of communication with PLC. If the PLC or OPC-server used supports the signal quality control, the communication quality flag may be input in a separate TRACE MODE channel. This channel value will be considered by the TRACE MODE for hardware validation/invalidation flag generation. Such technique helps developing the template algorithms for signal redundancy (or for groups of signals) in PLCs or in the fail-proof TRACE MODE servers.
Redundancy for SCADA/HMI Servers
 Reliability for TRACE MODE based SCADA/HMI PCs is assured by several network adapters support and by PLC communication lines redundancy. It assumes automatic switching-over of the server to standby network adapter in real time, in case of failure of the main adapter.
Reliability and fail-safety of distributed control systems, is provided by hot redundancy of control system servers – real time monitors, documentation servers and loggers. TRACE MODE 6 supports dual and triple redundancy features for servers as follows:
- project autobuilding for automatic replication of the redundant nodes in the TRACE MODE® 6 development system;
- automatic synchronization of real time data between the master and the standby servers;
- automatic switching-over of data flows to redundant server in case of failure of the master;
- automatic identification of server status as "master" or "standby" at system startup and automatic status conflicts resolution upon recovery of the master server after failure;
- logging of all failures and switchings to standby servers.
Like sensors, I/O cards and controllers redundancy, the most reliable and fail-safe is the triple redundant configuration of the TRACE MODE® 6 servers. No wonder that triple redundant systems is most frequently used for provision of reliability in aviation.
The following training video demonstrates how to make a simple redundant dual node control system using TRACE MODE DoubleForce RTM.
Download the project given in the film
Reliability and Redundancy for Clients
 A common weak point of the client-server architecture is dependence of operating availability of the entire control system upon the status of the server (in case of server failure, all the clients connected to it would cease receiving data).
The TRACE MODE 6 SCADA easily solves this problem by offering an up-to-date technology of reliability enhancement of client-server systems, by automatic switching of the client PCs over to standby server in case if the main server failure. Switching-over to dual, as well as to triple redundant servers is supported. In case of server failure, the TRACE MODE SCADA client switches over to standby data source automatically, and in case if the standby server also fails, – to the next redundant PC.
Thus, there are no problems in the client-server reliability for the TRACE MODE based control systems.
Undoubtedly, development of reliable fail-safe control system with hot redundancy features is a complicated task, however, help is always offered to developer by TRACE MODE autobuilding technology and by the TRACE MODE 6 powerful debugging tools.
Reliability with, and without Redundancy
Methods of reliability enhancement are also available in the TRACE MODE® 6, for control system having no hot redundancy. First of all, there are two new mechanisms of correct recovery of data exchange between servers and controllers after: "sending to" and "reading from".
The "sending to " option provides sending of all control values to the recovered node, independently from when the values changed for the last time. "Reading from" initiates preemptive reception of all input channels from recovered node. Thus, actuality of current data is provided at all nodes, immediately after recovery of communication between them. Both the functions can be also used for data exchange between two servers, and for data exchange between the server and controller, and even for exchange between two SOFTLOGIC controllers.
Another tool for control system reliability enhancement is real time dumping. It consists in saving in a file the real time values of all channels of the given node. All real time data is saved in this file with a preset period of time, which data can be read by server during restart, as initial values of the channels. Thus, rebooting of the server would not practically affect operation of the control system.
The integrated TRACE MODE® 6 SOFTLOGIC-SCADA/HMI-MES-EAM-HRM system provides advanced technologies that ensure reliability and fail-safe operation of control systems. The TRACE MODE redundancy features are automated and quite easy to implement. |
