Cayuga Networks Rugged Engineering Process Principles
Rugged Engineering Process Principles describes the overarching development and evaluation principles that we adhere to at Cayuga Networks. This document describes at a high-level our current best practices. Our overall goal is to:
- Produce high-quality software that delights and amazes our customers
- To do so as rapidly and cost-effectively as possible rapidly
- Respond to a changing threat landscape with timely software software updates
These principles include:
The Codebase Must Always Work — “Don’t Break the Build”
We strive to conduct all development in such a way that the trunk code, as well as key release branches, are always in working condition. Here “working” means that the code will compile, and that the code passes all automated tests that the company has developed to date. In the event that someone checks in something that breaks this working status, we make it a top priority for the engineering group to get the code working again. Further development activity will not proceed on top of a broken codebase, because this risks introducing multiple errors, which will be much harder to debug together than separately.
No Manual Testing — Automate Everything
At Cayuga Networks, we eschew manual testing, particularly in the company’s formal test/release procedures. While, an automated version of a manual test takes longer to develop, once integrated into the company’s infrastructure, it will be used thousands of times on many future versions of the codebase. It will also help to ensure that the particular condition tested remains correct indefinitely.
Design Automated Test Suites to Work Everywhere
We design our automated test suites for maximal easy reuse. In particular, it we make Easy to apply the test-suites:
- To the trunk codes
- To any branch that an engineer has created
- In a development environment against non-checked-in code
- To a codebase installed at a customer site to ensure it is performing as expected
We also design the test suites so that they are easy to demonstrate to technical staff of customers and prospects highlighting Cayuga Network’s high levels of engineering quality and product security.
Automated Test Suites Designed to Run All The Time
The company’s automated test suites are designed for constant use. To the extent possible, we run performance-wise, testing on every check-in on the trunk and on release branches. This ensures that errors can be immediately detected and remediated. Tests that are expensive or lengthy are run on a regular, twice-daily cycle on trunk and key release branches.
No Check-In Without Tests
We make sure no code that extends functionality is ever checked in without new unit tests. These unit tests demonstrate that the functionality is working correctly in all important respects. They are also integrated into the company’s automated test framework. Furthermore, we make sure:
- Code that fixes a bug is never checked in without at least one new unit test case demonstrating that the bug is now fixed
- No code is checked in without first having run the appropriate company test suite on it and ensuring that all existing tests pass, as well as the new ones
Put Functionality in the Product, Not the Test Harness
Wherever there is a choice, put functionality in the product, rather than external test harnesses. That way, the functionality is available in field debugging situations—as well as during testing—and on any instance of the product anywhere.
At Cayuga Networks we strive to make the entire test-suite available in the product, and architect the product such that in-situ performance test replays can be done using customer traffic.