Crypto Wallet Security Rating Report. Key Insights and Findings

July 17, 2023 • by CER.live Team

Highlights:

• Open source wallets do not guarantee security. However, combining open source development with a third-party hosted bug bounty can significantly enhance wallet security profile.
• Most wallets skip full application audits and bet more on bug bounties for proactive vulnerability hunting.
• Integration with hardware wallets does not improve software application security, but hardware integration significantly reduces the risk of losing funds in the event of a hack. 
• Wallets with a larger user base typically invest more in security. However, when popular wallets disregard best practices, they inadvertently encourage hackers to exploit security vulnerabilities and drain more funds.

Wallets: More Than Just App

As crypto assets find their place in the minds of people around the world, the waves of hype and fear wash every aspect of the crypto industry. And while cryptocurrencies and NFTs have the main stage in the current environment, crypto wallets are often the first step of every crypto rabbit hole. 

Wallets occupy a critical yet understated role. They serve as the primary ambassadors to the crypto realm, guiding users through their inaugural transactions and beyond.

The wallet experience has a profound influence, setting the tone for the user’s journey. A user-friendly, responsive interface can build confidence, while a complex or glitchy one can foster frustration. Importantly, the wallet is also the user’s first line of defense, its security measures either reinforcing trust or inciting doubt.

Moreover, the wallet’s array of features, such as multi-chain support or integrated DeFi platforms, can shape the user’s crypto exploration. However, these must not compromise performance, as sluggish wallets can dampen user experience.

As the broader parts of the Internet are adopting some Web3 technologies, such as logging in with your wallet, we expect the importance of a good wallet only to increase, as it will store not only your assets but also the decentralized identity of a user. 

This is why it was vital for us to do a comprehensive study of what wallet can and can’t do, analyze some of the common conceptions about them, and create our own security rating based on the most important aspects of wallet security. We can gain insight into how popular wallets protect user funds from theft by examining their security features and comparing them with hack incidents. This helps compensate for the absence of consistent and trustworthy information on the topic. 

This study was conducted with the objective to unravel the intricacies of crypto wallet security, weigh the pros and cons of various wallets, and arm users and developers with data-driven insights that can enhance their decision-making processes. Our methodology incorporated an in-depth analysis of 45 wallets across 12 platforms. We explored a plethora of data points including key management approaches and restoration methods, security audit history, bug bounties, pentests, and incident records, among others. Another vector of our research went in analyzing various features that enhance user experience and looking whether they compromise wallet security or not.

In the end we’ve developed a security score that looks at security aspects of each wallet and ranks them on a scale from 0 to 100. Security metrics were assigned different weights depending on their potential impact on the overall security of a wallet. The details of our methodology can be found here.

Overview of Crypto Wallet Landscape

Fundamentally, wallets come in different forms, akin to species in a rich ecosystem, each adapted to survive and thrive in different environments. Not all wallets are made equal, as some of them play crucial roles in their respective infrastructure and can require demanding hardware specs to be operational. Others are lightweight and focus on expanding user experience with features such as fiat on- and off-ramps, token swaps and NFT management. Another way to categorize wallets is by looking at the way they store users’ keys to their accounts.

In our research, the primary categorization falls into three groups: Desktop, Mobile, and Extensions. We’ve also analyzed several wallets that allow you to create an account on their website, but there was not enough information to make an exhaustive conclusion, therefore we skipped this category for now. Each of these types represents a unique user experience and set of security features, tailored to the requirements of varying user habits and preferences.

Desktop wallets, for instance, are applications designed for the optimal performance on personal computers and consist of three platforms – Windows, Linux and macOS. They often offer extensive feature sets and robust security, considering the larger resource pool they draw from. However, being tied to a physical machine, their accessibility is somewhat limited.

Mobile wallets provide the advantage of portability and are often designed with a user-friendly focus, catering to a generation constantly on the go. Their convenience, however, comes with the inherent security challenges of a mobile ecosystem, where risks like device theft, hacking, or vulnerabilities in mobile operating systems can potentially compromise user assets.

Extension wallets occupy a unique position in this triad, offering a balance between the desktop and mobile experiences. They allow users to interact with decentralized applications directly from their browsers, blending convenience and utility. That said, they aren’t immune to the same range of threats as their counterparts, and their security is closely linked to the browser’s overall security.

To understand the comparative security of these wallet types, we designed a methodology that considers various security features in four broad categories:

• Audits and Pentests;
• Bug Bounties;
• Built-in Security Features;
• Past Incidents.

By calculating an individual score for each application on every available platform, we can derive the average security score per platform category, which revealed a fascinating pattern: Desktop Wallets scored 48; Mobile Wallets fared slightly better with 51, while Extensions, contrary to popular misconceptions, emerged as the most secure platform with an average score of 58 out of 100. In other words, extension wallets pay more attention to their security, while desktop applications are the most vulnerable

The common pitfall of both Desktop and Mobile wallets was the absence of thorough and relevant dApp audit or a pentest that would cover an application itself. We attribute it to the amount of updates an average app has, where each new update can disqualify the pentest made earlier. However, the one pattern that became clear is that wallets in general are less likely to go through a pentest or an audit, with only 14 brands out of 45 providing any type of audit whatsoever. The champion in this category is MetaMask that does diligent audits of their app and functionality. Pentests, however, are even more scarce, with 6 brands in total, and half of them being out of relevance by now.

When deciding where to store your funds, it’s crucial to check if the wallet has undergone a full application pentest or audit. Skipping this step is not recommended, as a thorough audit considerably lowers the risk of the wallet being hacked. Remember to pay attention to this important detail.

Atomic Wallet Hack: Sobering Reality Of Security Importance

The Atomic Wallet Hack of June 2023 was a stark reminder of the crucial importance of security in crypto wallets. Despite boasting an impressive one million downloads on Android and around 5 million users in total, Atomic Wallet fell victim to a sophisticated hack, resulting in the loss of an estimated $35 to $100 million of various cryptocurrencies. This theft affected approximately 1% of the wallet’s active users, with the most substantial portion of the stolen assets being Tron-based USDT. The unfortunate incident sends a poignant message on how users’ perceived security may not always mirror the actual security measures in place.

A look at the potential causes of the Atomic Wallet Hack reveals that, from insufficient entropy in key generation, fault attacks on key-related algorithms, possible transmission of keys to a centralized server, to a potential supply-chain attack, there were several aspects that might have contributed to this security breach. Yet, what’s most alarming is the fact that red flags were raised by security audit firm, Least Authority in 2022. The audit which was hidden by Atomic Wallet and later disclosed by Least Authority highlighted issues such as flawed cryptography, non-adherence to best practices, and insufficient documentation as significant risks. Despite these warnings, the vulnerability of Atomic Wallet was not fully addressed, and could lead to this high-profile hack.

Let’s put this event into perspective, comparing it with other similarly popular wallets.

Despite having a similar number of downloads as Atomic Wallet, the following wallets had considerably higher security scores according to our rating:

On the other hand, Atomic Wallet had a score of only 41 before the hack and an even lower score of 31 after the incident. This sharp contrast suggests a significant correlation between the security measures taken by developers and the robustness of a wallet’s defense against hacking attempts.

A deeper dive into our data reinforces this point. As per one of our hypotheses, there is a high correlation between the number of installs and the amount of security measures a company takes. All wallets brands with more than 10M users got to the top of our rating because they take a higher number of security precautions. For example, MetaMask has several protective layers:

• detailed explanation when signing transactions
• hardware wallet compatibility,
• several audits,
• a detailed bug bounty program.

Essentially, popular wallets tend to adopt more robust security measures to protect their increasing user base. This seems logical – a higher user base often corresponds to more significant funds to secure, more visibility, and consequently, more potential threats. It can also result in a positive feedback loop, with more secure wallets attracting new users in higher numbers than the less secure ones.

However, Atomic Wallet appears to deviate from this trend, having a significantly lower security score despite its popularity. Apart from the above-mentioned audit, they did not have hardware wallet support, had a self-hosted bug bounty with few details about the scope and possible rewards, and few other security compromises. 

This case study is a cautionary tale for every crypto user that wants to choose a crypto wallet, and an example of why we need to boost the competition in wallet security between developers by introducing a security rating.

To Share or Not to Share: Open v.s. Closed Source Wallets

The decision to go open or closed source is one that has profound implications for a wallet’s development process. To fully understand this dynamic, it’s essential to first delineate what open and closed source entail within this context. 

Our team has considered a wallet to be open source if the major part of its cryptography and key management are available to the public, facilitating a democratic ecosystem where anyone can review and potentially improve the codebase. 

The difference in security features between fully and partially open source apps is insignificant. Yet, fully open source wallets have a considerably higher risk of getting hacked. In our study, five fully open source wallets have experienced security incidents:

• Edge
• MyEtherWallet
• Electrum
• Ledger Live
• Trezor

In contrast, none of the partially open source wallets have been hacked.

Initial expectations might suggest a stark contrast between these open and closed source development models, with open source wallets potentially being more secure due to the transparency and collective intelligence that such an approach brings. However, the data from our research paints a slightly more nuanced picture.

Upon analyzing security incidents across open and closed source wallets on various platforms, we observed interesting patterns. Open source wallets showed an average incident rate of 13.87%, while closed source wallets had a slightly higher incident rate with an average of 18.48%.

We analyzed hack incidents across different platforms for both open and closed source wallets. In closed source desktop apps, two had security incidents in the past – Coinomi and Atomic Wallet. In closed source mobile apps, security incidents are far less frequent – only 1 of 12 was hacked. By contrast, 4 of 26 open source wallets have experienced a security breach.

While these percentages fluctuated significantly across platforms, they underscore the point that both open and closed source wallets face security challenges, irrespective of their developmental models.

These statistics serve as a reminder that transparency does not automatically equate to invulnerability. Open source wallets, despite their embrace of public scrutiny, still experienced a notable rate of incidents. This trend suggests that while having a community of developers vetting the code can help identify vulnerabilities, it doesn’t inherently guarantee a flawless security architecture.

The implications for users are clear: they should prioritize wallets that consistently demonstrate robust security measures, regular updates, and efficient vulnerability management over whether they are open or closed source.

Developers, on the other hand, have to keep in mind that diligent monitoring and robust security practices are vital, regardless of the codebase’s nature. Open source does offer the benefit of community review, but leveraging this advantage requires active management.

Bug Bounties: Wallet’s Best Friend?

Saying all that, there is one major difference between closed and open source wallets – the latter can significantly improve their security by participating in a third-party hosted bug bounty program.

Bug bounty programs, serving as contracts between companies and independent security researchers, have proven instrumental in strengthening the security of crypto wallets. Whether self-hosted by the organization or managed by a third-party platform like HackerProof or Bugcrowd, they provide an avenue for identifying and remedying vulnerabilities before they evolve into significant threats.

Interestingly, our data illustrate the much greater effectiveness of third-party hosted bug bounty programs. Among 45 wallets featured in our study, 23 run a bug bounty program. Among 9 wallets with self-hosted programs, 4 have experienced a security breach. By contrast, only 1 of 14 wallets with a third-party hosted program (MyEtherWallet) has been hacked. In the case of MyEthereWallet, the bug bounty program was launched a few years after the hack.

Third-party programs grant more protection because these programs benefit from a larger and more diverse pool of researchers and white-hat hackers (many working full-time), which increases the chance of finding bugs. At the same time, 45% of wallets with self-hosted bounty programs have experienced a security incident, emphasizing that the program’s success isn’t merely about its existence but also hinges on quality management and resource optimization.

A high-quality bug bounty program is characterized by its transparency and detailed description, including defined scope, vulnerability types eligible for rewards, rules, and clear reward amounts. The minimum and maximum bounties, especially for critical bugs, reflect a company’s dedication to resolving serious security issues. For instance, Coinbase’s bug bounty program, hosted by HackerOne, offers up to a staggering $1,000,000 for critical vulnerabilities, demonstrating an exceptionally high level of commitment to their platform’s security.

Role of Hardware Wallet Compatibility

As the digital assets ecosystem expands, additional layers of security emerge. Hardware wallets are one of the additional security layer. If a software wallet supports a physical device, it allow user to increase their chances not to lose their funds in case of an attack on a wallet. While these wallets are often touted for their superior security attributes, it’s essential to objectively evaluate their effectiveness based on data and statistics.

In the context of blockchain wallets, hardware wallet compatibility refers to a wallet’s ability to interact with a physical hardware device that securely stores the user’s private keys. This approach isolates the private keys from vulnerable online systems, thus potentially reducing the risk of hacks and unauthorized access to funds. Hardware integration enhances security by adding additional authentication method for all transactions made from a software wallet.

Our research tested two hypotheses related to hardware wallet compatibility. The first hypothesis postulated that hardware wallet support decreases the possibility of a wallet being hacked. Our data indicates that wallets with hardware wallet compatibility do have fewer publicly known incidents compared to wallets without such compatibility. Specifically, 13.79% of wallets with hardware wallet compatibility had incidents, compared to 18.69% for wallets without such compatibility. However, since the difference between the number of incidents is not significant, we can’t definitely prove that software wallets with this compatibility are inherently more secure.

The second hypothesis was that hardware wallet support decreases the likelihood of funds being siphoned from the user’s account. This was a much stronger assertion since it is the exact reason why people use physical devices in the first place. And the data showed that wallets with hardware wallet compatibility had significantly fewer publicly known incidents that resulted in users’ stolen funds, compared to wallets without cold storage compatibility (4.58% versus 16.62%).

These findings imply that hardware wallets, while not a silver bullet for security issues, do enhance the overall safety of a wallet, particularly in preventing unauthorized access to users’ funds.

Security Practices Across Different Platforms

In the last section of this report dedicated to security in wallets we decided to circle back to types of platforms, and analyze the least common features among wallets deployed on these platforms. The reasoning behind this topic comes from an idea that we should point at the gaps in wallet defenses, for the developers to see where they could improve their design.

When we looked at the least utilized security feature across different platforms, for both Desktop and Extensions it was interesting to see that about 1/3rd of all wallets do not give sufficient information about a transaction. A minimally sufficient description of a transaction includes:

• Receiving address
• Transaction fee
• Transaction amount
• A message that describes the data content of the transaction

In addition, users would benefit from the following information:

• What permissions they grant and to which address
• The amount of funds affected by the transaction
• Possible implications of the transaction

We consider the lack of transaction descriptions a security hazard because of the ever-increasing phishing and scam transactions that mimic real Web3 applications. Wallets should inform their customers about all the rights and give warnings if the transaction gives too many rights to the third party, especially considering that extensions are still the most popular choice to access decentralized applications.

It is rather concerning that only 44.76% of all mobile applications in our database prompt users to make a mandatory backup before initiating any other activity in the wallet. This approach could possibly be due to an emphasis on user experience. While this may be perceived as an inconvenience that hampers users’ seamless dive into the world of cryptocurrency, the self-custody of one’s seed phrase or private keys remains paramount in terms of security standards. We strongly recommend users create a backup as their first step when setting up a new wallet, and we urge developers to reconsider their design strategies in order to amplify the security of user accounts.

Another area of improvement in mobile application security is adopting hardware wallet compatibility, as only 55.26% of apps do so. We’ve already discussed the benefits of having this feature enabled in case of a hack, and since most of the wallets pledge user security as their priority, it would be much better if most of the wallets would provide this option.

In the end, we also looked at the overall number of features a wallet has and compared it with known incidents with the idea that the increased amount of features may lead to higher vectors of attack. Fortunately, this hypothesis did not find any ground with correlations close to zero across all platforms.

Feature-Rich Environments

Although the primary role of wallets is to store crypto assets securely, we should not forget about the features that affect user experience. In this chapter, we look at the most and least common features in wallets, as well as identify the most feature-rich wallets.

When it comes to token swaps, a fundamental feature facilitating the exchange of one cryptocurrency for another, we found it is widely adopted across all platforms. This ubiquity underscores the core role of token swapping in contemporary wallets. However, moving from intra-chain to cross-chain operations, we observed that Extensions lag behind Desktop and Mobile counterparts in providing cross-chain swap capabilities.

Even less common is the integration of cross-chain bridges, especially in Desktop and Extensions platforms. We see a distinct lack of integrated bridges, although by now it is obvious that the future of crypto is multi-chain. This pattern most likely can be attributed to the fact that bridges are still one of the least secure parts of blockchain infrastructure with about $1.5b stolen from the bridge exploits in 2022 alone. Despite that, an official bridge can greatly improve user experience by decreasing their fees and enabling more Web3 instruments.

The analysis of fiat on-ramps and off-ramps reveals a prominent disparity. While most platforms enable users to convert their traditional fiat currency into digital assets, only around half of these provide the converse option to convert digital assets back into fiat currency. We see it being a result of pressure from the financial regulators, who impose stricter rules on companies that allow users to move funds from crypto into fiat.

For people interested in NFTs, blockchain based games and using their wallet to access external dApps, a mobile version of a wallet would often be a preferred option, with extensions being in second place. Desktop wallets do lag behind in these features.

Last but not least, staking options are more or less equally integrated across all platforms. However, we see this aspect to be the one that requires heightened attention from the user side as staking can be divided into external and internal. With external staking, a wallet provides direct access to the staking infrastructure of corresponding blockchain. However, internal staking is something to be suspicious of, as it is often unprofitable for a wallet to do so, and they may use users’ funds to increase the revenue as well as risks for users.

Among all the applications we’ve studied, the most feature-rich wallets appeared to be MetaMask and ZenGo on Android, as well as Trust Wallet on iOS. Moreover, these wallets have one of the highest rating scores on our list. In other words, these apps provide the finest balance between security and experience, allowing users to utilize their assets to the fullest.

Conclusions & Recommendations

In the crypto wallet ecosystem, it is evident that the diversity of wallets goes beyond just the platform; it involves the underpinning security measures and software development paradigms as well. Our findings suggest that regardless of the wallet type or a platform – there’s a common shortfall across the board: the lack of adequate and relevant audits and pentests with the wallet app in scope. Regular and thorough security assessments are the cornerstone of any robust security framework, hence we recommend that all wallet developers prioritize integrating periodic audits and pentests into their development roadmap, considering the dynamic nature of the digital asset environment.

The symbiosis between open source software and third-party hosted bug bounties is particularly noteworthy. This combination leverages the collective intelligence of the open source community and the rigor of professional penetration testers, thus creating a resilient security layer. We recommend wallet developers, especially those adhering to the open source model, to not only consider implementing bug bounty programs but to opt for third-party hosted options.

As for the users, when choosing a crypto wallet, security should be paramount among your considerations. Firstly, assess the wallet’s reputation and history – wallets with a history of breaches may not be the safest option. Secondly, check for the full-app audit, pentests and/or bug bounties – this would give you an answer to how much attention developers pay to their app security. If you are a hardware wallet user – you can find the list of all apps with cold storage compatibility on our website. And as a rule of thumb, expect the most popular wallets to have the highest security rating. If it’s not the case – be vigilant, and maybe choose another wallet!