A Developers Guide to Safer Smart Contracts

Smart contract developers face the challenge of creating robust code amid immense incentives and stress. Throughout my career, I've adopted a principle that shapes my approach to blockchain development and helps me by making my life harder: “Trust no one, not even yourself.” This isn't just skepticism for its own sake, but a shift in mindset that redefines how smart contracts are designed, tested, and deployed. Adopting this means questioning everything and taking nothing for granted, adding more layers to your testing and monitoring process that are indispensable for creating safer DeFi applications.

In traditional models, protocols employ developers who operate under tight deadlines and product pressures. Despite their technical skills, their primary incentives come from their salaries and, occasionally, token or stock allocations that weakly align with the protocol's success. Once the features are pushed, the code is handed off to an audit company, whose researchers often operate in brief, time-boxed engagements. Audit quality hinges on trust in a firm's track record, and while many auditors strive to deliver the best analysis possible, their attention and efforts are not always aligned with the critical stakes involved. They receive their payment regardless of their results, and the true quality of their work can take months or even years to fully assess.

Meanwhile, black hats are incentivized by millions—if not hundreds of millions—of dollars. They are ruthless, relentless, and unrestricted by conventional boundaries. With unlimited time and the freedom to employ any tactic, these adversaries will stop at nothing to exploit your mission-critical smart contracts. Increasingly, nation-state actors with near-unlimited resources have joined them, creating a seismic imbalance. One side employs a handful of mildly incentivized developers and auditors working under severe constraints, while the other side wields an army of highly incentivized hackers equipped with advanced tools and strategies.

This asymmetry demands a radical shift in our approach. At Mento Labs, we have an uncompromising security mindset where testing, verification and sophisticated strategies are essential safeguards against these adversaries. “Trust no one, not even yourself.” Test, retest, and verify every component using various techniques, in every possible environment, under various conditions, and conducted by different people. Never take security for granted. Don't assume your colleague will catch what you missed, or that an auditor will catch what both of you overlooked. Don't assume it will “just work.” Your security depends on diligence at every step, and establishing a culture of proactive security. With this post we aim to share our mindset and techniques to empower developers to fortify their projects and ultimately outmaneuver those who seek to exploit them.

The Case for Paranoid Development

We believe embracing a "paranoid development" mindset is crucial. This approach involves a questioning of every aspect of the smart contracts: no assumption is safe, every line of code could be a potential weak point, and trust is essentially non-existent. In practical terms, this means that developers assume every part of the smart contract could fail. This isn't about being pessimistic—it's about being realistic and prepared.

Blockchain technology is inherently secure, but its applications, smart contracts, are crafted by humans and are as fallible as their creators. The immutable and decentralized nature of blockchain also means that errors are often irreversible and can have high financial consequences. Therefore, assuming that each component of the smart contract could fail forces developers to cover all bases—testing extensively under various scenarios and constantly validating the integrity of every element. This exhaustive level of testing is aimed at fortifying smart contracts against both known threats and potential unforeseen vulnerabilities.

For example, the accidental triggering of a self-destruct function in the Parity Wallet led to over $150 million in Ether being frozen, highlighting how a small oversight, without any threat actor, can lead to significant financial losses. These incidents teach us that minor errors can have outsized impacts, magnifying the need for a skeptical approach in development.

In recent years, the scale and sophistication of attacks on blockchain have escalated, with nation-state actors like the Lazarus Group. These actors have resources and capabilities far beyond typical hackers, enabling them to execute complex and highly strategic attacks. Over the last couple of years, Lazarus Group has been implicated in thefts amounting to billions of dollars from various protocols. These are not mere opportunistic breaches, but are well-coordinated assaults that exploit very specific vulnerabilities in blockchain applications and human nature. Such high-level threats underscore the need for paranoid development practices that not only anticipate common threats, but also prepare for complex, multi-vector attacks orchestrated by highly skilled adversaries.

Common Techniques

Unit and Integration Tests

Just like any other team, unit testing is the foundation of our strategy. These tests cover both the basic functionality and the edge cases of smart contract components, starting with clear, concise test cases for every function. This includes considering both normal execution scenarios and potential error states, ensuring a comprehensive coverage.

Unit testing should ensure that every write operation to storage and every revert condition is checked. It's essential to implement both positive and negative test scenarios, giving special attention to complex parts of the code and value transfer operations. Testing should also verify that expected events are correctly triggered, enhancing the smart contract’s traceability.

Beyond unit tests, integration tests play a crucial role in validating how different parts of the system interact with each other. This involves combining individual modules and testing them as a group to ensure the entire system operates seamlessly. For smart contracts, this means testing interactions between various functions and external contracts to catch issues that might not be evident in unit tests alone. Additionally, integration tests are essential for verifying specific use cases and user flows, ensuring that the smart contract performs as expected in real-world scenarios.

To maximize their effectiveness, both unit and integration tests should be integrated into the CI pipeline. This integration ensures that tests are run with every PR, helping to catch regression bugs before they can be merged into the codebase.

These tests also serve as a valuable tool for external developers, auditors, and security professionals. They use unit and integration tests to understand how a particular, perhaps obscure, function is supposed to behave. This clarity is important for auditors and bounty hunters who rely on these details to validate the correctness and security of a contract. They can be also used as a base for PoCs to verify a potential finding.

Fuzz Tests

Fuzz testing is another important component of our security framework. This technique involves automatically providing random inputs to the smart contracts to check for vulnerabilities and bugs.

Through fuzz testing, we have identified and resolved numerous subtle bugs that traditional testing methods might miss. For instance, in our emission contract, we utilized the Maclaurin series to approximate the exponential decay formula. However, we overlooked a specific condition where, under some conditions and after a certain time, the curve of the approximation behaved unexpectedly and began to increase rather than decrease. This issue was missed during our unit testing. Fortunately, our fuzz tests highlighted this anomaly, allowing us to address the problem by incorporating an additional term into the approximation formula. Without the insights provided by fuzz testing, this issue might have remained undetected.

Fork Tests

Fork testing is a crucial element of any comprehensive testing suite, offering insights into how a smart contract will function within the actual blockchain environment. By creating a replica of the live blockchain, developers can simulate real-world conditions to observe how their smart contracts operate when interacting with evolving contract states and external components. This method is essential for uncovering issues that might not be evident in isolated tests and for confirming that the smart contract will perform as expected in the real world.

At Mento Labs, automated fork tests are a key part of our development process. We maintain a fork test repository for our governance, locking, and token contracts, which allows us to conduct periodic and automatic checks. This ensures that our smart contracts function correctly under current blockchain conditions. Our testing tool performs detailed evaluations by simulating real world interactions with our smart contracts. This proactive approach helps us quickly identify and respond to external changes, maintaining the integrity and robustness of our smart contracts.

Advanced Strategies

Formal Verification

Unlike traditional testing methods, which simulate execution with various inputs to find errors, formal verification uses mathematical models to prove or disprove the correctness of a smart contract's code. This process is useful because it can guarantee that a smart contract behaves exactly as intended across all possible states and inputs, making it an invaluable tool for verifying critical blockchain functions that must not fail. Formal verification addresses the limitations of human error in traditional testing by providing a systematic, mathematical approach to detect flaws that might not be visible even in thorough test suits or manual reviews. It is particularly important for smart contracts that manage significant assets or perform key operational functions within a blockchain ecosystem, where errors can lead to catastrophic outcomes.

At Mento Labs, we use formal verification selectively, focusing on the most critical functions due to the time consuming nature of the process. We utilize tools like Certora for this purpose, which allow us to model and verify the correctness of crucial functions within our smart contracts. A prime example is the update to our stable token contract. This smart contract features functions like creditGasFees and debitGasFees which are essential because they allow the stable coins to be used as a gas currency on the Celo blockchain. Given the high stakes involved—where flaws in these functions could potentially disrupt the entire blockchain—we committed to formally verifying these aspects of the smart contract.

On-chain Monitoring

On-chain monitoring involves continuous observation of smart contract activities and state on the blockchain to ensure that they are functioning as intended and to quickly identify any anomalies or malicious activities. Having analytical dashboards is crucial as they provide a real-time visual representation, allowing developers and stakeholders to gain insights into the operational health of their smart contracts.

Moreover, setting up alerts for unusual activities or deviations from normal operational patterns is vital. These alerts enable rapid response to potential issues, minimizing the impact of any problems that may arise. This proactive approach to monitoring helps maintain the integrity of smart contracts, ensures the security of digital assets, and supports the overall stability of the blockchain ecosystem.

At Mento Labs, we leverage both popular tools and proprietary technology to ensure comprehensive on-chain monitoring. We utilize platforms like Dune Analytics for general contract monitoring and analytics. Dune Analytics provides customizable dashboards that help us track smart contract interactions, metrics, and other critical indicators that inform our operational strategies and security measures.

We have developed an in-house tool named Aegis that plays a pivotal role in our monitoring framework. Aegis is designed to expose the results of on-chain view calls as Prometheus metrics, which are then ingested into Grafana. This setup allows us to create sophisticated monitoring dashboards tailored to our specific needs. Additionally, through Prometheus alerts, we are immediately notified if unexpected activities occur. Aegis is generic and business logic agnostic, it can be adapted to monitor any smart contract effectively.

Circuit Breakers

Circuit breakers in DeFi are modeled after similar systems in traditional financial markets and software engineering. These mechanisms are designed to halt or limit trading activities temporarily in response to abnormal market conditions, thereby preventing cascading failures or severe impacts from sudden market volatility. Implementing circuit breakers helps maintain system resilience, protects market integrity, and safeguards user assets by providing necessary pauses during extreme market fluctuations.

At Mento Labs, we have implemented an automated on-chain circuit breaker within the Mento Platform to enhance security against abrupt price changes and price manipulation risks. This system is pivotal for managing risks, especially considering the critical role of oracles in providing necessary external data for the protocol's operations.

Our circuit breaker is centered around a smart contract called the BreakerBox, which oversees all related operations and maintains the necessary state. This smart contract interacts with core components of the protocol, such as the BiPoolManager, which manages asset pools, and the SortedOracles contract that stores exchange rate reports.

The BreakerBox is equipped with various safety mechanisms, including adjustments to trading modes based on real-time data and conditions determined by dedicated breaker contracts. Each breaker contract follows a set logic to evaluate whether a specific rate feed should change its trading mode, effectively allowing our system to react dynamically to market conditions.

For a more detailed exploration of how circuit breakers function within our system, readers are encouraged to refer to our two-part blog post series on the topic:

• DeFi Security: Inside the Mento Protocol's Circuit Breaker

• DeFi Security: Mento Circuit Breaker in Action

These posts provide comprehensive insights into the design, implementation, and operational nuances of the Mento Platform’s circuit breakers.

Conclusion

At Mento Labs, we apply a multifaceted security strategy, embodying the essence of paranoid development. We continuously test and retest our smart contracts using a variety of techniques to ensure they are fortified against potential threats. Our framework encompasses unit tests, integration tests, fuzz tests, formal verification, on-chain monitoring, and circuit breakers, each contributing a layer of scrutiny that challenges our code from different angles. With the diverse toolset we use daily, we verify everything and trust nothing.

This post marks the first part of our series focused on the efforts and techniques we employ at Mento Labs to secure our smart contracts. In the upcoming second part, we will dive into how we engage with external security actors such as auditors, bounty hunters and audit contest platforms. We will explore our collaborative processes, how these relationships enhance our security measures, and what we expect from these interactions. Stay tuned to learn more about how external validation plays a critical role in our security strategy, reinforcing our commitment to maintaining a high standard of security.

Discover more

Payd and Mento Labs: Empowering the Gig Economy with Decentralized Stablecoins

November 5 2024

by Behice Uzun

Unveiling PUSO: The Launch of The First Decentralized Local Currency Stablecoin Tracking the Philippine Peso

September 25 2024

by Victoria Calmon

Highlights from Carbon DeFi’s Simulator Showcase

November 14 2024

by Behice Uzun