ciilf

The Core Infrastructure Initiative exists to support work improving the security of critical open source components. In a Linux system a flaw in the kernel can open up the opportunity for security problems in any or all the components – so it is in some sense the most critical component we have. Unsurprisingly, we have always been keen to support work that will make this more secure and plan to do even more going forward.

There has been some public discussion in the last week regarding the decision by Open Source Security Inc. and the creators of the Grsecurity® patches for the Linux kernel to cease making these patches freely available to users who are not paid subscribers to their service. While we would have preferred them to keep these patches freely available, the decision is absolutely theirs to make. From the point of view of the CII, we would much rather have security capabilities such as those offered by Grsecurity® in the main upstream kernel rather than available as a patch that needs to be applied by the user. That said, we fully understand that there is a lot of work involved in upstreaming extensive patches such as these and we will not criticise the Grsecurity® team for not doing so. Instead we will continue to support work to make the kernel as secure as possible.

Over the past few years the CII has been funding the Kernel Self Protection Project, the aim of which is to ensure that the kernel fails safely rather than just running safely. Many of the threads of this project were ported from the GPL-licensed code created by the PaX and Grsecurity® teams while others were inspired by some of their design work. This is exactly the way that open source development can both nurture and spread innovation. Below is a list of some of the kernel security projects that the CII has supported.

One of the larger kernel security projects that the CII has supported was the work performed by Emese Renfy on the plugin infrastructure for gcc. This architecture enables security improvements to be delivered in a modular way and Emese also worked on the constify, latent_entropy, structleak and initify plugins.

• Constify automatically applies const to structures which consist of function pointer members.

• The Latent Entropy plugin mitigates the problem of the kernel having too little entropy during and after boot for generating crypto keys. This plugin mixes random values into the latent_entropy global variable in functions marked by the __latent_entropy attribute. The value of this global variable is added to the kernel entropy pool to increase the entropy.

• The Structleak plugin zero-initializes any structures that containing a  __user attribute. This can prevent some classes of information exposures. For example, the exposure of siginfo in CVE-2013-2141 would have been blocked by this plugin.

• Initify extends the kernel mechanism to free up code and data memory that is only used during kernel or module initialization. This plugin will teach the compiler to find more such code and data that can be freed after initialization, thereby reducing memory usage. It also moves string constants used in initialization into their own sections so they can also be freed.

Another, current project that the CII is supporting is the work by David Windsor on HARDENED_ATOMIC and HARDENED_USERCOPY.

HARDENED_ATOMIC is a kernel self-protection mechanism that greatly helps with the prevention of use-after-free bugs. It is based off of work done by Kees Cook and the PaX Team. David has been adding new data types for reference counts and statistics so that these do not need to use the main atomic_t type.

The overall hardened usercopy feature is extensive, and has many sub-components. The main part David is working on is called slab cache whitelisting. Basically, hardened usercopy adds checks into the Linux kernel to make sure that whenever data is copied to/from userspace, buffer overflows do not occur.  It does this by verifying the size of the source and destination buffers, the location of these buffers in memory, and other checks.

One of the ways that it does this is to, by default, deny copying from kernel slabs, unless they are explicitly marked as being allowed to be copied.  Slabs are areas of memory that hold frequently used kernel objects.  These objects, by virtue of being frequently used, are allocated/freed many times.  Rather than calling the kernel allocator each time it needs a new object, it rather just takes one from a slab. Rather than freeing these objects, it returns them to the appropriate slab. Hardened usercopy, by default, will deny copying objects obtained from slabs. The work David is doing is to add the ability to mark slabs as being “copyable.”  This is called “whitelisting” a slab.

We also have two new projects starting, where we are working with a senior member of the kernel security team mentoring a younger developer. The first of these projects is under Julia Lawall, who is based at the Université Pierre-et-Marie-Curie in Paris and who is mentoring Bhumika Goyal, an Indian student who will travel to Paris for the three months of the project. Bhumika will be working on ‘constification’ – systematically ensuring that those values that should not change are defined as constants.

The second project is under Peter Senna Tschudin, who is based in Switzerland and is mentoring Gustavo Silva, from Mexico, who will be working on the issues found by running the Coverity static analysis tool over the kernel. Running a tool like Coverity over a very large body of code like the Linux kernel will produce a very large number of results. Many of these results may be false positives and many of the others will be very similar to each other. Peter and Gustavo intend to use the Semantic Patch Language (SmPL) to write patches which can be used to fix whole classes of issue detected by Coverity in order to more rapidly work through the long list. The goal here is to get the kernel source to a state where the static analysis scan yields very few warnings, which in turn means that as new code is added which causes a warning it will more prominently stand out, which will make the results of future analysis much more valuable.

The Kernel Self Protection Project keeps a list of projects that they believe would be beneficial to the security of the kernel. The team has been working through this list and if you are interested in helping to make the Linux kernel more secure then we encourage you to get involved. Sign up to the mailing lists, get involved in the discussions and if you are up for it then write some code. If you have specific security projects that you want to work on and you need some support in order to be able to do so then do get in touch with the CII. Supporting this sort of work is our job and we are standing by for your call!

This month marks the three year anniversary of the formation of the Core Infrastructure Initiative. It’s also the third anniversary of the Heartbleed vulnerability that served as a wake up call for the industry and which was a catalyst for the creation of the CII. For those not immersed in the security or technology industries, that bug revealed just how widespread and critical open source software is to the Internet’s infrastructure. The simple yet damaging security vulnerability uncovered in the hugely popular OpenSSL software had an enormous impact, in some cases allowing attackers to steal passwords, private keys, credit card numbers, financial information and more. At the time, it was estimated that almost one in five secure web servers were vulnerable to attack.

That episode also exposed limitations to Linus’s Law “many eyeballs make bugs shallow.” While in theory the openness of open source allows for huge numbers of people to get involved in checking the source, when software lacks an investment commensurate with its importance, we’re all at greater risk.

To help correct this, the Linux Foundation mobilized to form the Core Infrastructure Initiative. Twenty industry giants, including many of the world’s largest software companies, joined us in our initial mandate to secure the projects that are most critical to businesses on the Internet. To achieve this we set out to identify projects at risk, understand their needs and provide them the resources necessary to both make them more secure in the short term and stay more secure in the long term. As reportedat the time in The Economist, “OpenSSL, with its single main developer scraping by without a fair salary, was highlighted as a project that needed most attention.”

Less Fire-Fighting, More Strategizing

So three years in, are open source software vulnerabilities still as big a problem? Has the awareness raised by Heartbleed had a positive impact on online security and open source management? What have we been able to to do the make things better?

Firstly, of course software vulnerabilities are still a problem, in open source and in closed source. Sadly this is likely to be the case for quite some time. As long as software is specified by and written by humans this isn’t changing anytime soon. That said, we have made tremendous progress in the last few years.

Heartbleed uncovered a major gap in how we protect and secure the technology we use everyday. It showed us there is a major need to build a pre-emptive and collective system, absent of any one company’s individual priorities, to safeguard the Internet today and into the future. Quantitative and qualitative analysis of security of software, both closed and open, helps safeguard corporations and individuals.

I’m proud to say CII has made real progress and achieved many of our initial objectives, including our goal to make OpenSSL significantly more secure. Funding from CII has facilitated the fixing of many of its bugs and importantly reduced the chance of introducing new bugs. The OpenSSL team has indicated that it has moved from being in “firefighting mode” and are now more actively able to pursue strategic approaches to securing the project. Static and dynamic analysis are regularly performed as well using dynamic fuzz testing tools like AFL. In a few weeks time, we will be releasing results from an external audit of the OpenSSL code base that CII funded. The OpenSSL project now has a well-defined and published approach for how it will informs all interested parties of security advisories. The project is more secure than it was three years ago, both in terms of the code and the process, and we are delighted to have been instrumental in helping this happen.

Beyond OpenSSL, the CII has provided direct funding for architectural, development and testing work for dozens of other projects, relieving some of the financial pressure felt by many of their developers and allowing them to reduce technical debt and make structural improvements that will pay dividends into the future. Details of many of the successes from 2016 can be found in our most recent annual report.

Early on we recognised that we needed to apply quantitative and qualitative measures to find where risks lay. Our first CII Census project used a variety of metrics around bug density, developer community engagement, the number of security vulnerabilities and download and usage statistics to help identify open source components might be sources of risk and help target these for support. I am pleased to say that the CII members recently voted to extend funding the Census project to allow it to expand the number of packages under consideration, draw more detailed usage data from more sources and provide more continuous updates so that we can track projects as they improve (or, hopefully rarely, get worse).

Another success is the CII Best Practices Badge, which uses a qualitative self-assessment approach whereby open source project participants can grade themselves against a set of Best Practices for Open Source Development. Since formally launching in May 2016, more than 700 projects have signed up for the process and more than 70 projects have earned the badge.

We specifically reached out to both smaller projects, like cURL, and bigger projects, like the Linux kernel, to make sure that our criteria made sense for many different kinds of projects. The list of projects that proudly display the badge continues to grow — GitLab, Node.js, OpenBlox, OpenSSL, OpenStack, OPNFV, and Zephyr. The CII Badges program continues to evolve with work underway to introduce new badge levels to provide more sophisticated criteria.

Going forward we see the CII needing to do less fire-fighting and being able to apply more strategy. While we don’t expect to see an end to the need for supporting important maintenance work and to underwrite “orphaned projects,” many of our most successful initiatives have been the ones that have allowed us to help hundred or thousands of open source projects, rather than supporting them one at a time. The Best Practice Badge has helped hundreds of projects review and improve their security process. The Fuzzing Project has also applied dynamic fuzz-testing tools to hundreds of projects, while the Reproducible Builds project has helped enhance the build systems of tens of thousands of projects. We are also supporting the ongoing development of open source security testing tools ranging from the OWASP ZAP project to the Frama-C static analysis tools

Maintaining the code that we depend upon is still very important but we also need to build systems that allow us to help a much wider open source community. Thus, while our initial mandate was to target the projects that are most critical to businesses on the Internet, the CII is targeting the broadest range of projects possible within this remit — established and new, large and small, infrastructure and front-facing — in order to make the biggest impact possible. Below is a chart that shows our spending pattern over the last year. As time has gone by, the CII funding is moving up and to the right as we assign more funding to projects with continued high impact. We expect this trend to continue.

The first graphic titled “Annual Investment in Project” shows the current state of confirmed spending through 2017. The second one, titled “Total Investment in Project,” illustrates CII spending since our start three years ago.

New Structure Expedites Funding Decisions and Grant Dispersal  

As we enter our fourth year, the CII has also made some changes to its membership structure. We need to be able to expand our membership, and we need to be able to make decisions quickly. To that end, earlier this year we updated our charter and and introduced new membership levels, creating a smaller, elected Steering Committee (SC) and a new Investment Committee (IC). With these changes, CII’s committees are more empowered to make swift decisions related to the organization’s operations and distribution of funds. Additionally, the CII charter now explicitly calls out the Steering Committee’s role to provide governance, oversight and audit of the CII and the role of a separate Investing Committee that will determine funding of specific projects.

Having two committees with more distinct areas of focus also means CII members are able to nominate someone from their legal and/or Open Source teams to work on governance issues, and appoint someone with domain expertise to vote on grants and funding decisions.

CII also introduced Platinum and Gold membership levels. The only difference between them is whether the member gets automatic representation on the CII Steering Committee (for Platinum members) or gets to vote to elect SC representatives (for Gold members).

Aside from now having an elected steering committee rather than direct representation, CII has also changed the way in which we vote on which projects we want to fund. Previously we needed to have a majority of members vote in favor of a project. Now we open voting for a period of three weeks and require a majority of votes cast in that window in order to accept the proposal. Voting can also close early once more than 50 percent of members have voted one way or the other. With these changes, all members are able to have their say, but never hold up the voting process. We believe that these streamlined procedures will allow us to get the resources where they are needed more quickly and also ensure that when great open source developers are available we can snatch them up quickly before they take a job elsewhere.

We’re proud of the progress we’ve made in the past three years. We took on a huge and open-ended challenge. By its very nature we will likely never be “done,” but it is clear that

we have already made a significant impact. Going forward we will continue to build better open source security tools, drive better security processes and support the communities that are building the technology on which we all depend. We will also continue to support many of the teams that toil to maintain the old foundation stones of the Internet, some of which go back decades.

We look forward to the next three years!