Workflows, Data Management & Automation
Large collaborative projects often require coordinated integration and processing of complex data through multiple pipelines. These pipelines may share common components but must operate effectively in different contexts and use cases. CAC consultants work with researchers to build and support workflows for data integration, management, and downstream analysis, aiming to automate processing in high-throughput scenarios.
Sample Projects
- SCiMMA: Multi-Messenger Astronomy Infrastructure
In the SCiMMA project, CAC computational scientists lead a multi-institutional team developing a cloud-based messaging system. This system coordinates low-latency, distributed workflows across multiple scientific teams and instruments around the world.
- NANOGrav: Gravitational Wave Detection
As part of the North American Nanohertz Observatory for Gravitational Waves (NANOGrav), CAC scientists built a robust data management system that automatically collects, organizes, and archives data from multiple radio telescopes. The system shares datasets for processing and ensures that final data products are accessible to the scientific community.
- Fred Young Submillimeter Telescope
CAC consultants are contributing to the development and analysis of data pipelines and workflows for the
Fred Young Submillimeter Telescope which will see first light in 2026 and will further insights into the structure of the universe.
- Digital Agriculture & Farm of the Future
CAC scientists and consultants are collaborating with Cornell researchers to develop software systems that ingest, organize, and process data from a wide range of sensors and monitors tracking dairy cow health and milk production. This work includes the creation of the
MyCow$ tool to support decision-making on dairy farms, as well as software solutions to integrate data from the Cornell Agricultural Systems Testbed and Demonstration Site (CAST) for the
Farm of the Future.
Ready to Collaborate?
Contact us — we’re happy to help.