Partial upgrades to existing automation systems can be very difficult, requiring close cooperation between the system integrator and plant personnel.
Stephen Goldsworth, Systems Engineering Group Manager, Raju Nair, PLC Programming Manager, and Jonathon Shores, a member of the Systems Engineering Group at Tesco Controls Inc. wrote an article for the October 2018 issue of Applied Automation titled Complex Upgrades Demand Advanced Expertise. Here’s a summary, click on the link above for the full text.
Extended Downtime is Not an Option
When it comes to upgrading automation systems, some of the most difficult projects are partial upgrades where some components must be replaced while others need to be left in place, often for budgetary reasons. For many of these projects, extended downtime is not an option, increasing the degree of difficulty. Water and wastewater plants are often particularly susceptible to these issues because of aging control systems, limited budgets and requirements for continuous uptime.
Close Partnership
The authors say that projects involving partial upgrades require a very close partnership between the end-user and the system integrator throughout the project, from design to startup.
Inevitably, issues with existing hardware and software, and the integration between old and new, are found after the contract is signed. Only a true partnering arrangement, where both parties are committed to each other for the long term, can allow each of these issues to be addressed with little or no impact on the project cost, schedule, and performance.
The two water/wastewater projects described below each fit that description, and each was completed successfully through a partnership between a municipality and Tesco® Controls Inc., a system integrator specializing in the water/wastewater industry.
Project 1: Replacing PLCs, Converting Protocols, and Upgrading SCADA System Hardware and Software
A ground water treatment plant was running two different Wonderware® InTouch® (v. 9.5) SCADA applications on obsolete Windows® XP computers. Each was communicating to in-plant Square D® SyMax® PLCs over a legacy serial SyNet® communication protocol and media. Several remote SyMax PLCs were also connected to the SCADA system over a serial radio link.
The municipality’s ultimate goal was to replace all legacy PLCs at both in-plant and remote sites, but the project was budgeted for a phased approach, with only three PLCs initially slated for replacement.
In the first phase, one in-plant PLC and two remote PLCs would be replaced with a hot-standby Schneider® Modicon® M580 PLC and Schneider Modicon M340 PLCs, respectively. The remaining plant and remote PLCs would be replaced in future phases. However, the municipality needed to keep the other existing in-plant and remote SyMax PLCs communicating and operating during the upgrade process.
The plant also wanted to upgrade the SCADA software and hardware to Wonderware InTouch for System Platform 2017, running on a virtualized infrastructure. One of the main goals of this project was to upgrade plant operations by improving PLC maintainability and SCADA robustness, while keeping the existing plant operations intact. Outdated and unsupported physical-world liabilities were replaced with a new, virtualized SCADA system, with backups of all virtual machines as simple data which can be copied off to a storage device.
This significantly improved the end-user’s disaster recovery capability. The obsolete Windows XP computers were replaced with Microsoft Hyper-V cluster-based Microsoft 2016 Datacenter Edition virtual images.
With a modern and up-to-date SCADA system, the end-user no longer had to worry about where to procure the parts to support a generation of obsolete operating systems and control software.
Project 2: PLC Consolidation and SCADA Software Upgrade
In this municipal wastewater treatment plant, a SCADA system using Rockwell Automation® FactoryTalk® View SE (v. 7.0) communicated with over fifteen Allen-Bradley® PLC-5®s used for process control. Several other PLCs, primarily Allen-Bradley MicroLogix® models, were used for additional control. The SCADA system communicated to a ControlLogix® gateway via EtherNet/IP, and this gateway converted the protocol to DH+ to communicate with the PLC-5 network.
The end-user wanted to combine all of these existing PLCs into three hot-standby pairs of Allen-Bradley ControlLogix PLCs, each operating remote input/output (RIO) devices that communicated over the plant’s fiber optic EtherNet/IP network.
Tesco leveraged Rockwell Automation conversion products and tools to migrate both the Allen-Bradley hardware and Rockwell Software. Designing with the hardware retrofit kit allowed Tesco to maintain the original PLC-5 I/O terminations, and to couple these terminations with harnesses to the new ControlLogix hardware.
Through good documentation practices, appropriate contingency plans and a thorough understanding of the plant’s operational characteristics and the end-user’s goals, Tesco was able to perform a relatively seamless hot cutover of the PLCs. Tesco was able to cutover one PLC per day, with approximately twenty minutes spent for the physical hot cutover, and the remainder of the day used for testing and verification.
Conclusion
The authors say inexperienced system integrators may underestimate the full scope and complexity of these types of projects. This can lead to numerous change orders and delay requests as issues arise during project execution.
End-users with complex upgrade projects should instead seek an experienced and informed system integrator with a history of developing and maintaining long term relationships with their clients. This will allow these types of projects to be executed on time and at budget, while meeting or exceeding all performance requirements.
Authors
