The next higher-ed devops meetup will be March 2nd from 2PM to 3PM eastern time. Our meeting will be available on zoom: https://cornell.zoom.us/j/99442464891?pwd=eFdWMWRMTzRjTWJLOENJaVBkcmN6Zz09
Agenda
Michael B. Klein Northwestern University Library
Northwestern University Library’s Repository & Digital Curation workgroup has been developing and running cloud-hosted applications since 2016. Our initial approach was a simple “lift-and-shift” operation, moving our premises-hosted services to virtual machines hosted in Amazon EC2. We quickly migrated to some cloud-native utility services like our relational database and search index. More recently, we have started to take advantage of the scalability offered by cloud-native services such as serverless functions (AWS Lambda), serverless container infrastructure (AWS Fargate), and short-lived, large-scale batch operations.
Now we are taking the next step, translating our bespoke data processing pipeline into a series of state machines using AWS Step Functions and other cloud technologies. At each step, we have had to reevaluate our local development environment, either emulating or mocking the AWS services our applications depend on. To overcome the limitations of this approach, we have created a cloud-based development environment based on a heavily customized AWS Cloud9 environment. This has allowed each developer on our team to access the full range of AWS offerings, generating quick prototypes and iterating quickly on solutions, without the overhead or uncertainty of trying to emulate the entire stack on a laptop or local workstation, but without sacrificing the development tools we’re accustomed to. It has also led to a development platform that can be stood up and torn down quickly for easy onboarding as well as a quick “reset” of one or many existing setups. This presentation will explain our approach, how we got here, and how it’s going, as well as the surprises and challenges we have encountered along the way.
Slack Channel link: https://higher-eddevops.slack.com/join/shared_invite/zt-161fnj59w-GS09Y2wkO4yp5y~GqGJDpQ#/shared-invite/email