Python’s PyPI reveals its secrets

ReportApril 11, 2024Hacker NewsSoftware Security/Programming

GitGuardian is known for its annual Secret State Spread report. In a 2023 report, they found more than 10 million passwords, API keys, and other credentials exposed in public GitHub commits. Key points in the 2024 report go beyond highlighting the 12.8 million new Some secrets have been exposed in GitHub, but some are hidden in the popular Python suite repository PyPI.

PyPI, short for Python Package Index, hosts more than 20 TB of files and can be used for free in Python projects. If you have ever typed pip install [name of package], which most likely pulled the package from PyPI. Many people also use it. Whether it’s GitHub, PyPI or others, “open source suites account for approximately 90% of the code running in production today,” the report states.” It’s easy to see why these kits help developers avoid reinventing millions of wheels every day.

In 2024 report, GitGuardian reports discovering over 11,000 vulnerabilities Unique In 2023, PyPI will add 1,000 secrets. That’s not a lot compared to the 12.8 million new secrets added to GitHub in 2023, but GitHub’s is orders of magnitude larger.

What’s even more distressing is that nearly 100 of the secrets launched in 2017 are still valid 6-7 years later. They do not have the ability to check the validity of all secrets. Despite this, more than 300 unique and effective secrets were discovered. While this is a bit shocking to the casual observer, and doesn’t necessarily pose a threat to random Python developers (contrary to the 116 malware packages reported by ESET in late 2023), it does pose a threat to the owners of these packages. Said it was a threat of unknown magnitude.

Although GitGuardian has hundreds of secret detectors, it has been developed and refined over the years, and some of the most common secrets detected in overall research in 2023 were OpenAI API Keys, Google API Keys, and Google Cloud Keys . It is not difficult for a competent programmer to write regular expressions to find a single universal secret format. Even if there are many false positives, automated checks to see if they are valid can help developers find little treasure troves of exploitable secrets.

It is now accepted logic that if a key has been published in a public repository such as GitHub or PyPI, it must be considered compromised. In tests, honeytokens (a “teethless” API key that cannot access any resources) were tested for validity by bots within a minute of being posted to GitHub. In fact, honeycoin is like a “canary” to more and more developers. Depending on where you place a specific honeycoin, you can see that someone has been snooping there and get some information about them based on the telemetry collected while using the honeycoin.

When you accidentally release a secret, the bigger concern isn’t just that malicious actors might increase your cloud bill. This is a place they can go. If an over-permissioned AWS IAM token was compromised, what might a malicious actor find in the S3 bucket or database to which they were granted access? Could a malicious actor access other source code and corrupt something that would be delivered to many other people?

Whether you submitted a secret to GitHub, PyPI, NPM, or any public source code collection, the best first step when you discover that a secret has been compromised is to revoke it. Keep in mind the tiny window between honeycoin release and utilization. Once a secret is revealed, it will likely be copied.Even if you do not detect unauthorized use, you must Suppose an unauthorized malicious actor now possesses it.

Even if your source code is in a private repository, stories abound of malicious actors gaining access to private repositories through social engineering, phishing, and of course, leaking secrets. If there’s a lesson in all this, it’s that the secrets of plain text in source code will eventually be discovered. Whether they are accidentally released publicly or discovered by someone with access they should not have, they will be discovered.

All in all, no matter where you store or publish your source code, whether it’s a private repository or a public registry, there are some simple rules you should follow:

1. Don’t store secrets in plain text in your source code.
2. By strictly limiting the privileges granted by these secrets, those who possess them are prevented from exploring.
3. If you find yourself giving away a secret, undo it. It may take you some time to ensure that your production systems have new, undisclosed business continuity secrets, but revoke it as soon as possible.
4. Implement automated features like those provided by GitGuardian to ensure you’re not relying on imperfect humans to perfectly adhere to best practices around secret management.

If you follow these, you probably won’t have to learn the hard way that the owners of 11,000 secrets went through when they posted them to PyPI.