Beyond the Endless Frontier

Recommendations for NSF's Tech Labs initiative

The National Science Foundation’s Directorate of Technology, Innovation, and Partnerships (NSF-TIP) recently issued a request for information on a new “Tech Labs” program. From the RFI:

“Tech Labs will support full-time research, development and innovation (RDI) teams focused on overcoming persistent barriers to the commercialization of emerging technologies. These teams will benefit from operational autonomy, milestone-based funding and the ability to engage across academia, industry, national laboratories, and nonprofit sectors.”

Getting this program right is critical for the future of ambitious research in the US. If it’s successful, other agencies will hopefully copy and riff on it. Like so many new approaches to research funding and management, Tech Labs could go one of two ways: it can unlock a whole world of ideas that wouldn’t have seen the light of day, or it can experience mean reversion and capture by incumbent institutions.

We submitted the following response:

Which types of teams and organizations should be considered eligible to apply for the NSF Tech Labs program? What restrictions on team eligibility should be in place to maximize speed and ensure novel impact?

First, let’s enumerate the thing that would blunt tech labs speed and novel impact:

• Employees, especially leadership, who are simultaneously employed at other organizations after

  phase 0. This is especially true for professors and grad students. A fuzzier line is a situation where the leadership has some explicit agreement that they have a job waiting for them when the tech lab ends, like a professor on leave. This latter situation is also hard to enforce. One common important-to-avoid problem is the professor who is officially on leave, but in practice is still supervising a lab’s worth of grad students “on the side”

• If the tech labs org was a subsidiary of a large organization that has the ability to dictate things like research priorities, how money is spent, and how IP is assigned.

• If the people in the tech lab were spending a lot of their time doing outside fundraising and mixing a lot of money that has different requirements. This is another tricky one: there are definitely ways to leverage private funding, but seeking it out and aligning it all in the same direction is a huge distraction (especially during the tech labs period).

• If the lab did not own its core IP (ie. it was licensing it from a university or company.)

• If the lab has already raised significant amounts of venture capital or federal funding. Of course, what is “significant” is an open question. This one is tricky, because there are many projects that are a good fit for the tech labs initiative that have raised venture capital to get started because it was the only option. At the same time, tech labs work should not just be a way of subsidizing the venture capital industry.

In order to avoid these traps, we recommend the following restrictions:

By the end of phase 0, tech labs need to be incorporated as an independent organization with its own board of directors, and staffed by a full-time leadership team who have left their previous roles along with a majority of employees. The organization should have received less than $2M in grants and $1M in VC funding. Its IP should not create entangled relationships with external organizations.

These suggested funding limits and independence requirements are extreme. They would rule out several organizations that absolutely should get tech labs grants. For example, Convergent Research’s FROs are structured as fully-owned LLC subsidiaries. Similarly, many great organizations that would be great uses of Tech Labs money have received significantly more than $2M in grants – places like FROs, Otherlab, and Blueprint Biosecurity to name a few great orgs. Speculative Technologies ourselves would be ineligible!

The reason for these perhaps overly-restrictive requirements is that there is no other way to write the rules in order to prevent a reversion-to-the-mean/Matthew effect scenario that I worry is quite likely. If subsidiary organizations are legitimate, what’s to prevent somewhere like the Broad Institute or a university from spinning up a subsidiary subject to the same bureaucracy and incentives that (I hope) Tech Labs is meant to bypass? If they’re eligible, legacy institutions look much better on paper (track record! Institutional capabilities! brand!) and it would be hard for decision-makers in the NSF to justify overlooking them for newer, scrappier organizations. The same argument goes for funding limits. There is no rule that can distinguish between a well-funded FRO that nevertheless could do so much more and an SBIR-farm that turns government money into reports.

Some things that tech labs should be able to do:

• Hire some part-time employees (who could be grad students or postdocs) as long as they are a small fraction of the total employees.

• Use core facilities or rent space in a university, as long as they retain organizational independence and have the arrangement figured out by the end of phase 1.

• Hire contract research organizations, vendors, or subcontractors to do some of the work, up to 40% of the budget.

• Work with a fiscal sponsor.

Is the proposed timeline for Phase 0 (9 months), Phase 1 (24 months), and Phase 2 (24+ months) well-calibrated to support the program’s strategic objective of achieving high impact, accelerated outcomes? If not, what adjustments should be made and why?

The timescale seems well calibrated, assuming there is a way of calibrating the milestones at the end of Phase 0 for where the team was at the beginning of the effort. That is, there should be different expectations for a team that started off fully part time without facilities and one that was already working full time with a lab.

One potential problem with the proposed timeline is that it will be incredibly hard for phase 0 tech labs to start hiring technical and operational staff before they have phase 1 funding committed; otherwise they might all be laid off within a few months!

How should IP rights be structured to support maximum success and impact?

Any IP created during the tech lab should be owned by the tech lab entity, but the entity should be required to license that IP via a standard, non-exclusive license to any American entity. This approach balances incentivizing the creation and commercialization of IP and public good from taxpayer money. This approach to IP was one reason why Bell Labs was able to be so impactful. Anything outside their core telecom scope had to be licensed with generous terms, which allowed several different entities to do follow-on innovation on things like the transistor, cell phones, solar cells, and unix. More generally, impactful system-level innovations require the mixing of many pieces of IP, so it is important to incentivize sharing.

There are some domains where commercialization may be impossible with non-exclusive licenses. The current Tech Labs effort has explicitly said that they will not be funding therapeutics, the primary domain with this situation, but there may be others and hopefully the concept expands to other agencies that do deal with therapeutics. (It’s also worth remembering that health technology is weird.) In these situations, IP should be exclusive only within a particular field of use and the burden of proof should be strong that exclusive licenses are absolutely necessary for translation.

Tech labs should also encourage a culture of openness and trying to benefit an entire ecosystem, not secrecy and value-maximizing. The latter would be more likely if there was a potential lucrative IP prize waiting for successful tech labs. From a more hard-nosed angle, one of the reasons the Chinese hardware ecosystem has been able to flourish is because their healthy disrespect for IP enables a lot of tinkering by many parties and drives down prices through competition. The US should not loosen IP law, but tech labs should strive to replicate those productive conditions in an American context.

A tricky but likely common situation will occur when a tech lab is built around already-existing IP: a team spinning out of a university, for example. As table stakes, all licensing agreements need to be settled by the end of phase 0 (and ideally beforehand), otherwise things become incredibly messy. If at all possible, those licensing agreements should give the tech lab org ownership of any derivative IP and not put any conditions on its use (so that it can be non-exclusively licensed as mentioned above).

What degree of independence is optimal to ensure the flexibility, freedom, and speed required for the Tech Labs initiative? How should NSF define team independence?

Let’s flag two kinds of independence: team independence and organizational independence. Team independence is independence with respect to NSF TIP as specified in the funding agreement; organizational independence is independence with respect to all other entities and contracts.

Some prerequisites for the flexibility, freedom, and speed for Tech labs to maximize impact:

• The ability to spend money as quickly and flexibly as possible. Obviously there are federal rules on how money can be spent and the money needs to be audited at some point, but there should be explicit effort to make the requirements as loose as possible.

• The ability to shift technical approach as long as the concrete goal remains the same.

• A minimum amount of written reports. One way to minimize reports is to encourage and accept capability demonstrations as acceptance criteria for milestones.

• Teams should be able to set the milestones that they will be judged on (potentially after iteration with the tech labs). There should be a mechanism to renegotiate these milestones if the team heavily pivots their approach.

• Minimize committees. Committees, especially committees of experts with no skin in the game, select for consensus good things and drive results towards the mean.

There’s an open question about how much independence teams should have over the goal/output of the tech labs effort. In theory, teams should be able to pivot towards a new goal if they find a more promising one during the course of the effort; however this could also lead to a lot of wasted money and floundering if the team finds that their original idea is no good and doesn’t want to shut down the org. It’s impossible to entirely prevent this possibility but ironically, the solution might be to have more, lower-stakes touchpoints with a program officer who can get a sense of whether the team is doing good work or not.

How should funding be allotted to each proposed Tech Labs? What factors – for example: team size, team expertise, infrastructure needs, growth trajectory – should NSF consider to determine appropriate funding amounts to support successful Tech Labs teams?

The NSF should not consider team size or expertise. Teams should have the incentive to be as efficient as possible: teams should be rewarded, not punished for figuring out how to be productive with a few, relatively junior people.

Funding should actually be allocated as team-agnostically as possible. Budgeting based on team need will incentivize teams to either pad out their budget to the maximum allowed if budget isn’t part of the funding decision or skew their budgeting needs too low if it is.

However, some work is more or less expensive than others. Animal studies are far more expensive than pure software work that doesn’t involve training AI models, for example. To address this, NSF should have a simple rubric that consists of work-requirements-based multipliers that get applied to a base budget. As a made up example, say that animal studies are a 1.2x multiplier, training AI is a 1.1x multiplier, and custom hardware is a 1.1x multiplier. Then a project that is proposing to build a custom piece of hardware to take measurements on an animal model and train a foundation model would be allocated 1.2x1.1x1.1xbase budget = 1.45 x the base budget.

What opportunities do you see for synergy with research and development efforts that are or could be funded by industry or philanthropic organizations? What partnership structure would allow Tech Labs to leverage federal and private support for maximum benefit?

Synergy between private and public capital sounds great, but can often lead to traps or perverse incentives in which winners win more and teams that are good at politicking and sales are rewarded over those with the most technical capability. There are potentially good ways for private and public capital to synergize in the tech labs efforts but it needs to be carefully thought through.

Some particular things to avoid:

• Requiring private matching in phase 0 or phase 1 or giving an advantage to organizations that already have funding lined up or in the bank.

• Allowing philanthropic or industry organizations to weigh in on which teams are funded.

One way to leverage philanthropic and industry capital in a way that avoids several downsides is to require or encourage matching funding for phase two. This would give teams the freedom to hone in on a direction and show results before needing to fundraise and ‘answer to multiple masters.’ It would potentially have the advantage of providing a “forcing function” towards a specific application or transition path once the system is at a point where that is beneficial.

Teams should certainly be encouraged to coordinate closely with industrial R&D teams in order to make sure that their work is tuned for real use cases, but requiring those relationships to involve matching funding could unnecessarily weed out good teams and cause a lot of trouble around IP and transitioning in a way that maximizes impact.

An implicit synergy between tech labs and philanthropic organizations could emerge over time if tech labs became a regular, common thing across the government: philanthropic efforts could “seed” projects that become well-situated for a tech labs effort. Today, philanthropic capital is often hesitant to fund ambitious independent research projects because of sustainability questions – they don’t want to be on the hook for the entire cost of the effort but also don’t want to partially fund something that will die without their support.

What translational problems, challenges and/or bottlenecks could be addressed within 3-7 years with this program design? Answers can be broad or specific.

Some potential problems

• Building a new manufacturing paradigm that would make producing physical products much more like creating software – easily reconfigurable, rapidly scalable, and with tight feedback loops between inputs and outputs. Creating the “chatGPT” moment for this new approach would require figuring out how to integrate into a single system robotics, ai-driven feedback loops, and newly available tools like advanced magnets, lasers, or software-driven machines.

• Creating the design and scalable manufacturing tools for protein-based fibers. If we were able to design the analogues of spider silk and manufacture them at scale, it could unlock new possibilities for robotics, microelectronics, and textiles.

One thing to flag that tech labs should avoid doing is being too prescriptive about specific challenges that will be considered in the upcoming call. The ideas that will enable the most novel impact from the tech labs will not fit neatly into tight topical buckets like “new metal processing” or “advanced solar cell manufacturing.” We strongly recommend broad topical areas like “manufacturing” “biomanufacturing” “energy” and allow teams to surprise you with the challenges they think they can tackle within them.

Miscellaneous Feedback

Some things to flag that were not explicitly asked about in the RFI:

Spinning up and running a new org involves a lot of hard work beyond the core technical aspects – especially ones like tech labs will support that do not follow any standard template. Teams will need a lot of help to hit the ground running. This includes both operational pieces like setting up all the functions of an org – HR, accounting, etc – and strategic/coaching help on how to manage a fast-paced independent org, plan and roadmap multiple coordinated technical thrusts, and transition technology in order to maximize the impact of the work. In order to maximize teams’ chances of success the tech labs initiative should curate and vet some set of core services. This set of core services could include outsourced HR/accounting and legal (like that provided by Convergent Research to their FROs), and coaching+strategic planning like the Brains research accelerator.

It’s likely that this call won’t fully address the “cold-start” problem. That is, if the program selects for teams that look like they have the best chance of getting the farthest during the tech lab, it is going to heavily favor pre-existing teams that have already gotten funding in one way or another to do as much work as possible. That precondition already puts a constraint on the novel impact that the tech labs can enable because there will be a selection effect for work that already looked promising to a more traditional funding source. It’s likely out of the scope of this initial effort, but in order to maximize the value of the tech labs program, TIP should also look at funding organizations that are able to start ambitious efforts “from scratch” to get them to a place where they would be a good fit for a full tech labs proposal. Another approach to this “cold-start” problem would be to accept a few earlier-stage teams into phase 0 based on potential and then judge them on whether they can make shocking amounts of progress in a short amount of time compared to other more mature teams.

It’s important to make successful phase transitions as deterministic as possible. That is, teams should have a good sense of what “good enough” to make it to the next phase looks like and feel like the decision is not arbitrary or out of their hands. Ideally, it would be as simple as “if you hit the aggressive milestones that you set, you’ll make it to the next stage.” The proposed budgets and number of teams in phase 0 make that ideal scenario unlikely, but it can still stand as a thing to get as close to as possible. This determinism is important to incentivize teams to genuinely put it all on the table. If it feels like a team can go all out, crush their goals, and still have their funding cut then they will spend a lot of their time and effort making backup plans and going on side-quests to prepare for that contingency instead of focusing on the primary goal.