New age transportation: Use of Big Data & IoT in Railways & Metros

The use of new and emerging technologies is leading to improved quality of services, new savings, enhanced resource utilisation and efficiency.

Metro rail
Metro rail


Railway and Metro systems have complex technologies, with a wide range of human actors, organisations and technical solutions. To control such complexity, a viable solution is to apply intelligent computerised systems. Industry 4.0 refers to the so-called fourth industrial revolution characterised by smart systems and Industrial internet-based solutions. The transportation sector, especially railways, has largely adopted Industry 4.0. 

The use of new and emerging technologies is leading to improved quality of services, new savings, enhanced resource utilisation and efficiency. It has also facilitated the development of new services and business models based on the capability of the industrial internet and the analytics capabilities of big data. Big data has the potential to transform the current state-of-the-art railway technology platforms into a network of collaborative communities seamlessly moving freight and passengers and delivering services in a planned way. The current trend of automation and data exchange is towards adopting and adapting the new and emerging technologies to achieve new levels of effectiveness and efficiency. Big data in railways comes from interconnected stakeholders, which provides intelligence to the Railway, Metro, MRTS and various other transport systems. 

The complete big data architecture includes cyber-physical systems, the Internet of Things (IoT) and Cloud computing, all of which work together to create ‘Smart Railways’. An application area generating considerable excitement with possibility of better O&M through self-learning and smart systems that predict failure, make diagnoses and trigger maintenance actions. These systems make high demands on data access and data quality and use multiple data sources to extract relevant information.

Use of Big Data & IoT in Railways

Big data analytics in Railway O&M is based on the use of advanced technologies to perform predictive analytics and make decisions based on the analysis of huge amounts of data. Providing O&M services involves data collection, analysis, visualisation, and decision-making for assets. The use of big data in O&M addresses a sort of common achilles heel in asset management that of status forecasting, commonly called prognosis. The estimation of the remaining useful life of an asset, in order to ascertain the probability of its mission accomplishment, constitutes the basis for any operation or maintenance service and, as such, is key to the success of any organisation. 

Big Data for O&M

The objective of big data in railways and metros is to enable predictive algorithms from heterogeneous data sources, scalable data structures, real-time communications, and visualisation techniques. Such research is challenging in infrastructure asset maintenance in the railway environment in three particular areas: railway system component degradation prediction modelling; railway infrastructure and vehicle maintenance cost prediction modelling; and infrastructure and vehicle condition monitoring. The objectives of big data analytics in the railway industry are as follows:

  • Create real-time predictive algorithms from heterogeneous data sources that cope with privacy preserved processing, feature and instance selection, discretisation, data compression, ensemble classifiers and regression models, and data spatial and temporal alignment.
  • Create scalable data structures based on cross-domain data source acquisition by means of a virtualisation layer between the data acquisition process and data analytics. This includes new solutions that combine new database capabilities to integrate heterogeneous data sources in a high-performance accessing system based on the cloud.
  • Enables big data communications using open interface gateways with monitoring systems providing timestamp and position synchronisation, heterogeneous communication support, including mobility and aggregation, and priority protocols for real-time transmission of information.

Positive effects of Big Data Analytics for Railway Networks

Big data analytics have the potential to influence several dimensions of the railway sector and can overcome organisational, operational and technical complexities, including economic and human effects and information handling.

Need for Big Data Technology

Traffic Management Systems (TMSs) comprise sub-systems, often with limited integration capabilities and non-standardised interfaces and display rules. In fact, the number and varied nature of assets makes the integration of data sources extremely difficult; therefore, the network asset status information cannot be widely understood or exploited to inform TMS decision-making. Even more challenging is the integration with other information domains such as maintenance related services, energy resources etc., as this must be done manually. In summary, O&M are completely disconnected in terms of incoming data sources and decision-making. The estimation of the remaining useful life (RUL) to check the probability of mission accomplishment by the asset constitutes the basis of any operations or maintenance service.

Effect of Big Data on Operations

Big data enables automated, interoperable, interconnected, and advanced traffic management systems; scalable and upgradable systems, using standardised products and interfaces that enables easy migration from legacy systems. Due to use of big data a wealth of data and information on assets and traffic status is created with information management systems adding the capability of now casting and forecasting critical asset statuses. The positive effects of being able to accurately forecast an asset’s status does not just provide benefits for maintenance planning, but also benefits other areas, for example, traffic management. The use of big data in railways leads to improvement in the following operational areas:

  • A standardised approach to information management and dispatching system enabling an integrated Traffic Management System (TMS).
  • An Information and Communication Technology (ICT) environment supporting all transport operational systems with standardised interfaces and with a plug and play framework for TMS applications.
  • An advanced asset information system with the ability to nowcast and forecast network asset statuses with the associated uncertainties from heterogeneous data sources.

Organisational effects of Big Data

The various organisational effects of Big data analytics in railways can be briefly summarised as under: 

  • Long-term needs and socio-economic growth: Big data develops a common methodology for improving infrastructure capacity, safety, and environmental impacts for the users and participating groups.
  • Smarter Railway Processes: SMARTness in transportation is closely related to operation and maintenance methodologies aiming for self-configuration, self-maintenance, and self-repair systems to maximise capacity and asset use by minimising shutdowns. Instrumented, interconnected, and intelligent assets that is maintained in a very different way.
  • System integration, safety, and interoperability: New O&M policies based on big data harmonises RAMS (Reliability, Availability and Maintainability) analysis and calculations across borders, leading to greater interoperability. Safety increases as a consequence of the increased reliability. In addition, there tends to be a common way to integrate systems, creating complex assets as system of systems, but in such a way that reliability is not affected by the complexity along international corridors.

Last but not least, big data analytics have potential benefits for energy and sustainability in the railway domain. Better O&M decreases the energy consumption and reduce the carbon footprint of assets, both rolling stock and infrastructure. The use of big data in the railway sector optimises operation and maintenance methodologies by taking an holistic approach, considering the entire lifecycle of the asset in a ‘cradle to the grave’ approach and contributing to the sustainability of the transportation system in a significant way.

Use of Big Data for Asset Management in Indian Railways

Indian Railways is an organisation of such epic proportions that words such as ‘gigantic’ and ‘enormous’ seems inadequate. With route length of 1,26,511km (Dec 31, 2021), Indian Railway (IR) carried 1.2123 billion tonnes of freight and 8.086 billion (808.6 crore) passengers during fiscal year ending March 2020, making it the world’s largest passenger carrier and fourth-largest freight carrier. What this also means, from an operational analytics standpoint, is a lot of data. However, Indian Railways currently lack abilities to use this data to derive accurate, contextual, and actionable insights for its business. 

Applications of Data Analytics in Railways

Data analytics technologies and techniques provide a means to analyse data sets and draw conclusions about them which help organisations make informed business decisions. Big data analytics is a form of advanced analytics, which involves complex applications with elements such as predictive models, statistical algorithms and what-if analysis powered by high-performance analytics systems.

There is a whole range of opportunities which IR can explore in the area of big data analytics. Some of these applications include customer experience, train scheduling, timetabling, improving security at railway stations, automatic charting, network optimisation, crew management, inventory management, and IRCTC ticket management. The existing data from the passenger reservation system, operating control, CCTV cameras at stations, maintenance depots and stores can be used intelligently to yield business benefits in the above areas. Current IT systems such as the real-time train information system (RTIS), the nation train enquiry system (NTES), and the control office application (COA) are some examples where data is being used to derive useful information.

One significant area where the use of data analytics is crucial is asset management. Asset management includes conducting an inventory of system assets, providing adequate staffing and training, performing preventative maintenance, and demonstrating adequate funding. In order to meet these requirements, an organisation must identify its needs and costs and develop long-range financial plans. This is called asset management planning. It consists of developing a plan to reduce costs while increasing the efficiency and the reliability of the assets.

Asset management in Indian Railways

Asset maintenance is part of part of asset management and involves condition monitoring of different assets through IoT devices. Indian Railways is a custodian of significant asset base. Indian Railways asset base includes approximately:

  • 126,511 Track Kms
  • 45,881 Route Kms of Electrified Track
  • 3,207 Route Kms of Automatic Signalling
  • 12,729 Locomotives
  • 2,93,077 Freight Wagons
  • 76,608 Coaching Stock
  • 7,500 Stations.

Indian Railways maintains huge terabytes of asset data. Using connected IoT sensors, historic data and analytics, the data can be turned into useful information to improve asset utilisation and availability. As of now, some efforts have been made to improve asset availability by adopting better maintenance strategies. These efforts have been made in departmental silos and lack a comprehensive approach. There is an opportunity to take a collective initiative to improve overall asset utilisation on Pan-India basis.

Existing condition monitoring and data collection systems in IR

  1. Signalling Systems: Through data loggers, the health of gears such as track circuits, point machines, axle counters and other internal circuits, is monitored and SMS are generated for various fault alerts. Furthermore, out of 7,500 stations, more than 5,500 stations have been equipped with Wi-Fi facilities. Though the capabilities exist, they are not utilised to the full extent. It is required that a more rigorous approach should be adopted to collect all kinds of asset data through sophisticated sensors and modern data analytics techniques be used to reduce maintenance requirements.
  1. Rolling Stock: Indian Railways is envisaging converting its ‘freight examination yards’ into technology-driven ‘Smart Yards’ for automatic detection of faults / defects / deficiencies in freight wagons. The automatic defect detection equipment of a Smart Yard shall provide advance data about hot axles and wheels, wheel flats, wheel profiles and diameter, load imbalance, loose and hanging parts, etc. even before the rake arrives at the maintenance yard. It will then use this information for objective fault assessment and proactive staffing, thereby, reducing turn-around time while boosting safety and improving productivity.
  1. Locomotives :  Remmlot enables health monitoring as well as fleet management of locomotives. The system monitors various operating parameters to achieve fuel saving and helps in analysis in case of untoward incidences. Electric locomotives have developed a remote diagnostic system (RDS). This GPS-based system monitors the current location of a locomotive and generates SMS-based alerts to maintenance staff with respect to isolation of key sub-systems. The system also has the capability to monitor traction motor voltage, status of auxiliary machines like compressor, traction motor blower, etc. However, the existing system generates alarms only on isolation.
  1. Over-head equipment (OHE) : Oliver ‘G’ is an over-head line inspection system with video recording and GPS tracking for improving the reliability of OHE. However, mostly manual inspections of OHE are done and various parameters are measured at regular interval unless there is any change in track parameters or there has been any OHE modification work.
  1. Track: The existing system for monitoring the health of the track includes OMS Car, TRC Run, Oscillograph Car and other field inspections. Indian Railways is working towards introduction of an Integrated Track Monitoring System that shall consist of:
  • Track parameters recording system
  • Full rail profile and wear measurement system
  • Rear Window Video Recording of track
  • Video recording of track components and analysis by image processing for status of track components
  • Other Transducers/Sensors.

Though several systems have been installed for data collection and condition monitoring for different assets, the analytics capacity of these systems is very limited. Furthermore, all these data collection systems work in technical silos, each with its own acronym, making it difficult to integrate data to acquire useful information.

Creating a data-driven Railway 

A lot of work is required to be carried out in the area of data collection, data management and data service provision so that the benefits of data analytics can be reaped for improved asset reliability. Some pointers are:

  • The creation of a supplier ecosystem with IoT-based capability
  • Data collection across all asset types and provision of new data collection equipment and platforms
  • The opportunity for managed services through a single service integrator
  • Integration of asset data, operation data, asset data and maintenance using adequate interfaces
  • Easy mechanism to exchange asset data throughout the lifecycle of assets
  • Decision modelling and risk analysis tools to identify bottlenecks
  • Raise maintenance alerts for different assets to enable safe, efficient and effective delivery of the operational railway
  • Strategic maintenance planning and creation of reports for easy accessibility of asset information.
  • Distribution of tablets to field maintainers and direct access to asset data and maintenance requests from central system through existing Wi-Fi facility, thus removing paperwork.

Indian Railways must absorb these huge banks of data and capture them in a national intelligence model; one that can serve this data-driven railway in a cohesive, safe and cost-effective manner.

Reasons for emphasis on IoT & Big Data

As the Internet of Things (IoT) and Big data gets more pervasive, industrial engineers are looking at new possibilities on how the technology can be leveraged for greater business impact. The rail industry is in a position where it can exploit the potential of industrial IoT and evolve without substantially increasing its investments in infrastructure.

Below narrated are five different ways in which IoT & Big Data can and has started redefining the railways and metro transportation bringing in increased efficiency and enhanced passenger experience:

  1. Greater Reliability and Safety: A train that suddenly breaks down on the track can ruin the day of its passengers, lead to delays across the network, and essentially throw the entire system off-gear. However, recent developments in preventive maintenance practices prompted by IoT & Big data have helped to revive the reliability of even the oldest assets. By integrating IoT sensors crucial components like brakes, wheel sets, and engines, trains become more sensitive to their operations for more efficiency. 

Maintenance of rail tracks also benefits from IoT. By deploying sensors across track systems, operators can be on top of track stress and conditions, temperatures, and other variables that have predictive values for maintenance teams. If problems with wagons and tracks can be identified proactively, the operators can take pre-emptive actions for safer operations.

  1. Fewer Maintenance Delays: Undesirable downtime due to sudden repairs can soon be a thing of the past for the railways. Predictive and preventive maintenance is feasible and more effective in the IoT era. Smart sensors and analytics across the train engine, coaches, and tracks allow rail systems to be remotely checked and repaired before a small issue magnifies into huge trouble. Asset health monitoring through IoT insights implies less of maintenance delays and helps in extending the life of rail infrastructure.
  2. Advanced Analytics for Streamlined Operations: The operators can control their trains more efficiently by tracking them across networks and processing the data using analytics. Some companies also employ IoT to check the flow of passengers—those waiting at the stations, travelling in each train coach, and the times when the passenger flow is the highest. Analytics on such data can guide operators on optimisation of travel schedules as per commuters’ needs and demands.

Weather also affects rail system in a region. It can impact the condition of rolling stock and its regular operations. The IoT savvy operators have started to incorporate predictive weather modelling in their operations to be ready for and avoid service interruptions caused by adverse weather conditions.

  1. Restructured and Optimised Passenger Experience: Consumers have fast adapted to digitalisation in the retail and banking space. The transport industry, including rail companies, is also transforming to meet passenger expectations with superior services. They offer e-tickets, scheduling information, and other solutions to travellers via smartphones and emails.

IoT can help take this experience a step ahead. It can help operators personalise the travel experience for individual passengers. For instance, services can be priced differently for different travellers as per the frequency of their travelling. Rail operators can enjoy greater passenger loyalty using IoT systems to understand customer experience history and make improvements for a more comfortable and convenient journey.

  1. Better Product Development in the Industry: Rail OEMs and operators can leverage IoT not only for better operations with the given infrastructure but also in the manufacturing processes of locomotives, wagons and train coaches. Conventional engineering solutions were not devised to support systems of systems. There can be delays and constraints in production when the process entails developing a requirements definition and then following it up with design, build, and tests. 

Feedback on manufacturing processes is an inherent part of product development with the Internet of Things concept. Engineers can use analytics with operations and performance data to derive valuable, actionable insights. This helps them understand the manufacturing procedures more dynamically and enhance final product’s quality sooner than in traditional methods. Continuous engineering and IoT can help to quicken the delivery of more sophisticated and connected products in the rail industry.

In a nutshell, IoT is bringing together two families of technology:

  1. Enterprise IT for improved resource planning, customer relationship management, and decision support systems.
  2. Operations technology to monitor and manage field equipment, production, and manufacturing processes

To be viable stakeholders and innovative contributors in the digital future, rail companies will need to make essential changes in their strategies. And the time to push further with the IoT is now.

Smart Trains and Connected Railway

With the advent of autonomous vehicles and improved cargo management the industrial ‘Internet of Things’ has a major impact on the transportation industry. One area that has seen less coverage is the connected railway. The fact that trains operate at such high speeds through tunnels and extreme weather conditions, presents real challenges when it comes to deploying IoT systems. But advances in networking have made smart trains a possibility, and one that could provide significant benefits when transporting goods, providing comfort for passengers and increasing operators’ return on investments.

Legacy infrastructure is gradually being replaced by train management systems in which trains become interconnected communication hubs, transmitting data among themselves and to network control centers, and receiving instructions from control centers. Machine-to-machine communication, with some help from the cloud, enables operators to utilise equipment, tracks and stations more efficiently, while dramatically reducing safety risks.

Safety: Key area of Concentration

  • Safety of course is a primary element of IoT applications and solutions when it comes to train management. One safety use case is on-board train location and detection systems that enable trains to be ‘aware’ of the positions of other trains. This reduces the risk of collisions while allowing trains to operate safely in close proximity to one another.
  • Speed monitoring and control is another important safety application. Systems have been developed that can display train velocity for drivers and report speeds back to central control systems. On-board monitoring systems are interconnected with outdoor signalling systems that can regulate train speeds or even remotely command trains to stop based on track conditions, the positions of switches, the presence of other trains on the track and other factors.

Rail and Metro experts across globe arguably state that there are three major systems within railroads that automation and the IoT can bring significant benefits: Signalling, Interlocking and Level Crossings Control.

Signalling Systems control the movement of a train by remotely adjusting train speed and braking. More traditional signalling systems are based on radio-frequency identification along the train track, but wireless train to ground signalling is getting more and more common in both railroad and metro systems. Interlocking avoids conflicting movements on the tracks at junctions and crossings by using red and green light signals. The interlocking system works in conjunction with the signalling system to prevent a train from getting a signal to proceed if the route is proven to be unsafe. The IoT can further improve the system’s level of automation and its integration with the signalling system. Level crossings control is the third system that impacts safety on railroads. Studies reveal that IoT can help decrease the accidents related to level crossings significantly deploying cameras and sensors for increased safety.

Other Benefits

  • The automation of toilets can significantly reduce the cost incurred by the train operator and, at the same time, provide a better service to passengers who will less likely find a toilet out of order. Currently, most train operators are unable to determine the status of the on-board toilets in real time and a significant amount of manual checking is required.
  • Management of the video recordings on board. Many rail operators have to send personnel on board their trains to manually pick up the hard drive when video recordings are requested by a law enforcement agency for investigation of an incident.
  • Food and drinks can be easily refilled at the upcoming station if data is available in real time regarding the items sold.
  • Temperature can be remotely controlled to avoid issues with refrigerators that might not be working at all times but whose temperature is critical to preserve the food quality over time.
  • Predictive and preventive maintenance can dramatically increase the percentage of times a train is in use rather than sitting in a maintenance or repair shop, and also improve the passenger experience and safety.

Summary & Conclusion

The Internet of Things (IoT) is a term widely used in today’s discussion about emerging technological trends. Definitions vary, but common characteristics include:

  • Intelligent devices with unique identification codes
  • Communicating with multiple devices
  • Devices include electronics, software, sensors, and potentially actuators
  • Operation within the (Internet-) network infrastructure
  • Real-time analytics 

For the railway and metro industry, this is not a new concept; elements of IoT are integrated into every modern train with multiple control units managing technical systems while communicating with each other. Examples include the mechanical and electrodynamic brake system, and the train control unit as a ‘master’ of the information infrastructure in a train. This is true for both train-based and wayside systems. However, in the past, the focus has been predominantly on the function of the individual sub-systems. Rarely have the information processing systems on the train been leveraged as a source of valuable information and insights.

This had significant consequences:

  1. Sensors were deployed sparingly, only as far as necessary for the individual system function.
  2. Data collection acted only as a support for maintenance crews for fault-finding purposes.
  3. Only recently have trains begun collecting and communicating information to the wayside for further use to a larger extend.
  4. Flexibility and the possibility for upgrades have been neglected. This makes addition of an intelligent system on a train—or even a few additional sensors—an effort-intensive exercise for integration and (re-)certification.

What makes the concept of IoT in railway systems even more challenging is the access/ exposure to the internet, which may leave rail systems vulnerable. However, it must be remembered that internet access in rail systems must be allowed only under well defined and controlled frameworks to ensure seamless safety and security. 

The increasing use of the Internet of Things (IoT) has profound implications across industries including the railways. Sensors, devices, systems, and applications are integrated on smart networks and work in a collaborative and cohesive railway ecosystem to enhance passenger safety, improve asset reliability and efficiency, and lower capital and operating expenses. The shift from legacy infrastructure to building a holistic, cloud-based train management system is the way forward for railway organisations if they are to use assets, tracks, equipment, and stations resourcefully and significantly bringing down safety threats.

Potential Impact of IoT on Reliability and Safety

Achieving an increase in reliability and safety parameters by even a few percentage points is a rare statistical event, given that both these factors are already performing at very high levels. Despite this high-performance rate, incidents where a train must be taken out of service or is delayed, may create problems. Passengers may not be able to reach their connections in time or may be delayed in reaching their destinations. Given the domino effect of a single delayed train, extensive rescheduling may have to be undertaken across the whole network. To some extent, the current schedules maintain reserves to accommodate such delays, which means that the infrastructure is not used as efficiently as it could have been and the service to passengers is not as good as it should be. On the safety side, while trains generally offer one of the safest modes of transportation, there is a consensus that every person harmed is one too many, and that safety must improve continuously. Hence, IoT should not be used to collect data on accidents, but to collect data based on which the probability of accidents can be reduced. Some areas where a further investigation for IoT-based solutions might be fruitful to improve reliability and safety include:

  • Monitoring of failure-prone systems on locomotives, such as the engine or electrical systems can increase the reliability significantly.
  • Supervision of mechanical systems such as running gear and track. The failure of mechanical systems causes several hundred deaths per year, which could be significantly reduced. Collecting acceleration data from bogies will, in many cases, make the identification of potential track failures possible
  • Train doors could be monitored to see if they are properly closed. However, this would require operational changes as well, since passengers often leave doors open or even cling to the outside of the train in case of overloaded trains.
  • Warning systems (light/acoustic) in case a train nears areas, which are prone to accidents with people crossing the tracks.
  • Monitoring of bridges regarding material stress or dynamic behavior to detect changes indicating future failure
  • Monitoring the speed of trains by GPS-driven speed measurements. Evaluating the speed profiles to validate the adherence of drivers to speed limits, but also to have real time train location to optimise traffic.

Major Challenges in enhancing the efficiency and competitiveness of Railways:

  1. Deploying consistent Safety Standards
  2. Ensuring streamlined asset availability
  3. Reducing greenhouse gas emissions
  4. Mitigating operating and maintenance costs
  5. The augmenting capacity of the rail network

The various segments of rail operations that IoT and Big Data Analytics can improve:

  1. Machine to machine communication
  2. Signalling systems
  3. Wayside Communication
  4. Level Crossings
  5. Station Information
  6. Endpoint security

Best Practices for deploying IoT and Big Data in Railways

  1. To integrate IoT to make it future proof
  2. Identify areas for IoT and Big Data implementation
  3. Demarcate what aspects can be outsourced
  4. Partner with an experienced service provider

Various advantages of using IoT and Big Data 

  • Improved Operational efficiency
  • Enhanced Automation
  • Higher Safety Levels
  • Better Passenger Experiences
  • Reduced Risk of Downtime

The creation of smart, environment and user-friendly mobility systems is among the high priority directions in the evolution of transport worldwide. Rail transport is recognised as a vital part of this process. Meanwhile, radical advancement in the business environment, facilitated by ICT technologies, requires the existing business models and strategies adopted by rail operators to be brought up to date. The thorough understanding of the concept of digital transformation is paramount in the development of rail transport in the New Economy. 

Digitalisation, as an ongoing process of convergence of the physical and virtual world, is bound towards cyber-physical systems and is responsible for innovation and change in multiple sectors of the economy. 

The main technologies and solutions which have accelerated digital transformation in the railway sector in recent years are:

  • Internet of Things (IoT)
  • Cloud Computing
  • Big Data Analytics (BDA)
  • Automation and Robotics. 

The adaptation to the new conditions of the digital economy is visibly marked by the emergence of the concept Industry 4.0 as well as, recently, Railway 4.0 and Digital Railway. Mobile applications, e-ticketing, digital train control, signalling and traffic management, digital platforms for predictive maintenance are the key areas of digitalisation in the rail sector. 

New products and services are becoming an integral part of the operations of railway undertakings, infrastructure managers and manufacturers for the industry. As such they contribute to the creation of added value for multiple stakeholders in public transport initiatives, which facilitate the implementation of new concepts of mobility.

Metro Rail News is conducting a 2nd Edition InnoMetro 2022 on 28-30 April 2022, virtually focusing on Seamless Mobility. Join InnoMetro 2022 for a detailed discussion on the topic “New age transportation: Use of Big Data & IoT in Railways & Metros”.

Join as a delegate:

Join as a Speaker:

Join as a Partner:

Daily Updates from Metro & Railway


Please enter your comment!
Please enter your name here

This site uses Akismet to reduce spam. Learn how your comment data is processed.