Engineering Excellence Behind ECI's Real-Time Election Results Portal

The real-time data ingestion and substantial traffic on the Election Commission of India's (ECI) website shows their impressive BigTech-level engineering to manage the live counting process seamlessly. This data doesn't appear by magic.

Live counting is happening today, involving the collation of 700 million votes from 3 million Electronic Voting Machines (EVMs), which are then mapped to 543 constituencies and thousands of wards. For selected wards, Voter Verifiable Paper Audit Trail (VVPAT) auditing occurs live as well.

This task is not easy and involves batched updates, typically every 10-30 minutes, before reflecting on the final frontend pages. Multiple aggregation steps precede this, starting at the booth level, then progressing to the ward level, and finally to the constituency level.

When I say "real-time," I mean the entire exercise is live—not that the frontend website is directly connected to a stream from the EVMs. Data is not being ingested one day, aggregated another day, and then uploaded later.

The data isn't piped in real-time from ingestion, at least not to the visible channels; instead, it's cached for several minutes. Typical processes include:

• Push events from the EVM to a write-ahead log (WAL) queue.

• Consumers aggregate these events and store them in an OLTP format with replication. This could be SQL persistent or use something like Redis, where the data, being minimal, essentially stays cached.

• The frontend queries this OLTP via an API proxy or a backend-for-frontend (BFF) setup.

Ensuring Data Integrity and Security

Given the sensitivity and significance of electoral data, the ECI has implemented stringent measures to ensure data integrity and security from the point of collection to the final display:

1. Automated Data Ingestion: EVM data entry is largely automated, reducing the risk of manual errors. Automated systems also speed up the collection process, essential for real-time result publication.

2. Avoiding Recounting Errors: To ensure that votes are not recounted or duplicated, each EVM upload is uniquely identified and logged. This prevents duplicate entries and maintains data consistency.

3. End-to-End Data Security: From EVM to backend systems, data transfer is encrypted and secured. This guarantees confidentiality and protection against tampering.

4. Role-Based Access Control: Access to the data at various levels - from data entry operators to administrators - is tightly controlled and monitored. This ensures that only authorized personnel can access and manipulate the data.

5. Data Erasure Protocols: Once data is processed and confirmed, specific protocols ensure that raw data from EVMs is securely erased. This step is crucial to maintaining data privacy and integrity post-election.

These processes ensure nothing breaks. Effective engineering involves using a CDN to front the data, updating it with each batch. A naive implementation would allow frontend queries to hit the DB directly. It's these seemingly mundane details that enable scale, more than any flashy "10x engineering" efforts.

Key questions need to be addressed for complete understanding:

1. How is data ingested into the ECI system, automated or manual?

2. How do they ensure no data from EVMs is recounted?

3. What are the end-to-end data security measures from collection to storage on the EVM to backend reception?

4. How is role-based access control of data managed?

5. How is the data erasure ensured once it has been consumed?

Without answers to these questions, it ultimately comes down to effective caching and cloud scaling. For instance, the CoWIN portal was built using DynamoDB, and it’s likely the ECI system relies on some public cloud service as well, though this remains speculative.

While the CDN aspect is relatively straightforward, the secure transfer of data from millions of EVMs and mapping it correctly across thousands of constituencies is complex. This data needs to be 100% consistent before it's available, unlike product analytics where some events can be lost.

This Is Engineering Excellence...

#election2024