How AI is helping Nvidia improve U.S. Postal Service delivery

Nvidia this week detailed a partnership with the U.S. Postal Service to transform the latter’s mail operations with AI. According to Nvidia, its machine learning resources and tools enabled the Postal Service to process over 20 terabytes of images a day from more than 1,000 mail processing machines using 195 edge servers.

In 2019, the Postal Service had a requirement to identify and track items in its over 100 million pieces of daily mail. Data scientists at the organization thought they could expand an image analysis system developed internally into something broader, with edge AI servers strategically located at the Post Office’s processing centers. The hope was that the system would enable the Postal Service to analyze billions of images of mail and share the insights quickly over the network.

Recruiting half a dozen architects at Nvidia and other companies, the Postal Service arrived at the deep learning models it needed after a three-week sprint. The work was the genesis of the Edge Compute Infrastructure Program, a distributed edge AI system that’s running on the NVIDIA EGX platform at the Postal Service today.

Open source software from Nvidia, the Triton Inference Server, acts as a sort of digital mailperson between the edge servers, delivering the necessary AI models on demand. According to the Postal Service’s analysis, a computer vision task that would have required two weeks on a network of servers with 800 processors can now be accomplished in 20 minutes on the four NVIDIA V100 Tensor Core GPUs in one of the edge servers, HPE Apollo 6500s.

Model serving

Triton automates the delivery of AI models to different Postal Service systems that may have unique configurations of GPUs and CPUs supporting deep learning frameworks. An app that checks for mail items alone requires coordinating the work of more than half a dozen deep learning models, each checking for specific features.

Departments across the Postal Service, from enterprise analytics to finance and marketing, have spawned ideas for as many as 30 apps for ECIP, Nvidia says. One would determine if a package carries the right postage for its size, weight, and destination. Another would decipher a barcode, even in the presence of damage.

The plan is to get several new AI-powered apps up and running this year. Nvidia and the Postal Service say the barcode model could be on ECIP as soon as this summer.

Next-gen OCR

Seeking further improvements to its mail processing pipeline, the Postal Service put out a request for what could be the next app for ECIP — one that uses optical character recognition (OCR). In the past, the agency would have bought expensive new hardware and software or used a public cloud service, which takes a lot of bandwidth and has significant costs. This time, leaning on Nvidia expertise, the company deployed an AI-based OCR system in a container on ECIP managed by Kubernetes and served by Triton.

In the early weeks of the pandemic, operators rolled out containers to get the first systems running as others were being delivered, updating them as the full network was installed. Nvidia was awarded the contract in September 2019, started deploying systems last February, and finished most of the hardware by August.

The new solutions could help the Postal Service improve delivery standards, which have fallen over the past year. In mid-December, during the last holiday season, the agency delivered as little as 62% of first-class mail on time — the lowest level in years. The rate rebounded to 84% by the week of March 6 but remained below the agency’s target of about 96%.

The Postal Service has blamed the pandemic and record peak periods for much of the poor service performance.

“The models we have deployed so far help manage the mail and the Postal Service — it helps us maintain our mission,” Todd Schimmel — the manager who oversees Postal Service systems, including ECIP — said in a press release. “It used to take eight or 10 people several days to track down items, now it takes one or two people a couple of hours. This has a benefit for us and our customers, letting us know where a specific parcel is at — it’s not a silver bullet, but it will fill a gap and boost our performance … We’re at the very beginning of our journey with edge AI.”