Today, we’re announcing a new request for applications (RFA). Letters of intent (LOIs), the short statements that precede full applications, are due no later than 11 October.

It seems timely, therefore, to update a column that I wrote about a year ago describing how we, the members of SFARI’s science team, make decisions on research proposals. Although the fundamentals remain unchanged, there have been a number of notable changes as a result of advances in the field and our evolving view of SFARI’s mission and priorities.

All applicants are required to send in an LOI. These are short (three-page) statements that are intended to save time — for applicants and SFARI science team members — compared with the effort required for full applications. In the past few years, we have received about 400 LOIs in each RFA cycle. We invite roughly one-fourth of those applicants to submit full proposals.

Applicants should familiarize themselves with the two types of awards that we offer, Pilot Awards and Research Awards, and should carefully consider which option is more appropriate given the state of their science and the extent of preliminary data.

At the LOI stage, we read applications primarily with an eye toward whether they are in line with SFARI’s mission and whether and how they fit in with our existing portfolio of awards.

Each LOI is read by at least two members of the SFARI science team, which includes our new director, Louis Reichardt, plus Marta Benedetti, Wendy Chung, Alice Luo Clayton, Alan Packer, Julia Sommer and myself, with additional advice from our former scientific director Gerald Fischbach. We seek advice from outside experts if we need to, but by making this first-level decision in-house, we are able to streamline the process significantly.

Judging from previous years, the vast majority of LOIs will describe interesting work. It would be great if SFARI could fund all of them, but that’s simply not possible. For example, a few years ago the LOI requests totaled $310 million and our budget was (and remains) in the range of $18 million for new grants each year. (Investigator-initiated grants make up roughly one-third of SFARI’s yearly budget of about $60 million.)

Clearly, we have to make tough choices about which applicants to invite to submit full proposals. We know it can be challenging to describe work in three pages. If we feel an application is not likely to be competitive for one reason or another, however, we prefer to tell the applicant early, rather than after he or she has invested more effort.

Although our decisions at this point are mostly binary (either yes or no to a full application), we do advise some applicants to consider submitting more streamlined full proposals, for example by reducing a request for a SFARI Research Award to a Pilot Award.

How we interpret which applications are likely to best support SFARI’s mission is of course nuanced and evolves as new results emerge. But there are a few general guidelines that have emerged from previous RFA review panels, and from multiple internal and external discussions, that influence our decisions. I outline those below.

Genetic landscape:

We have supported (and continue to support) a number of large-scale efforts aimed at defining the spectrum of autism genetics. These have primarily centered on gene discovery in the Simons Simplex Collection (SSC), a repository of samples and data from families that have one child with autism and unaffected parents and siblings. The entire collection of approximately 2,800 families will soon be analyzed genome-wide at the level of copy number variations and exome sequences, and the data will be made available to the research community. (Whole genomes of a small number of participants with autism will also be completed.)

Prospective applicants in this area of research are encouraged to familiarize themselves with these results, and to think carefully about how their research would complement our existing and ongoing efforts. Given our extensive investment in the SSC and the informatics platform to support it, we will rank gene discovery efforts in other autism groups less favorably unless there is a compelling reason.

With hundreds of genes implicated, the research community will need to replicate results from the SSC and other collections in larger populations, prioritize genes using innovative methods (bioinformatics and others) and, importantly, integrate all of this information in novel ways to gain a complete picture of genetics’ contribution to autism spectrum disorders.

Overlap with targeted RFAs:

In addition to our most recent RFA (RFA 2013), we are in the midst of making decisions on two targeted RFAs that we also launched in 2013: Circuit Dynamics and Analysis of Simons VIP Biospecimens. We do not encourage submission of grant applications previously submitted to either of these RFAs unless the grant application has been substantially revised or unless there is encouragement from the SFARI staff to submit a specific application to the annual RFA. In the coming months we may also be launching other targeted RFAs in areas that we consider special priorities (community input is always welcome). In rare circumstances, we may pull applications from the general RFA and suggest that they be resubmitted to a more focused RFA.

Which mouse?

We receive a large number of applications that propose to use rodent models to explore molecular mechanisms underlying autism spectrum disorders. In September 2012, my colleagues wrote a thorough piece on SFARI’s plans for mouse models. We strongly encourage all applicants who propose to use rodent models to read it, as it summarizes our efforts to prioritize, behaviorally characterize and distribute mouse models. We believe that making well-characterized mouse models readily available is key for attracting top talent to work on autism.

Where’s the cohort?

We often receive applications proposing clever new ways of measuring various genetic, biochemical, psychophysical or neuroimaging parameters in people with autism spectrum disorders. Given that autism is a behavioral diagnosis, we strongly encourage efforts to forge links between the behavior and the biology.

The trouble is, the first step for many of these investigators is to assemble a study group. We’ve learned from previous grants that recruitment and retention is far more difficult and costly than many researchers anticipate. So, unless there is a compelling reason for a new cohort, we recommend that researchers collaborate with existing teams that already have a study group, and pay careful attention to ascertainment biases that can influence the results.

We encourage investigators to use structured cohorts such as the Autism Genetic Resource Exchange, the SSC and the Simons Variation in Individuals Project (Simons VIP), in which multiple types of data (medical history, psychometric testing, genetic, brain imaging, etc.) can be layered to create a coherent picture, and, importantly, be shared with the community to accelerate research. Some families that were recruited to the SSC ended up not being eligible for inclusion because of some of the criteria, such as having another family member with a psychiatric illness. In the cases for which biospecimens were collected before the exclusion was flagged (and some other circumstances), these specimens are now available in an ancillary collection.

One more thought on cohorts: Genetic discovery is moving fast, and the potential for connecting genotype with phenotype seems especially promising. So our view is that it is no longer sensible to study individuals who have autism without knowing something about their genetic background, or at a minimum having their blood in the freezer for future analysis. We will work with investigators who need advice about logistics on this subject.

Relevance and reach:

The issues in autism research are complex, and deciding which projects are most relevant to our mission is one of our greatest challenges. One might argue that autism is a result of altered brain function, so any project that aims to understand the brain — especially a brain system whose function has been linked to autism — is relevant.

But SFARI’s funding is finite, and to fulfill our mission we need to maintain a tight focus. The time seems ripe to bridge different levels of understanding in a push for a more coherent picture of autism, from genes to circuits to behavior. For example, if a particular cellular or circuit deficit is apparent in one mouse model of autism, is it generalizable to other backgrounds or genetic models?

Autism is highly heterogeneous, so documenting points of convergence for cells, circuits and systems will be critical. For work in the clinical realm, we favor research that is grounded in biology. Over the long term, we hope this approach will lead to transformative and effective interventions. We recognize the value of other types of work, such as research into improvements of behavioral therapies or assistive technologies, but have generally prioritized studies that focus on elucidating biological mechanisms.

Clinical and translational proposals:

We encourage proposals aimed at development of good outcome measures and biomarkers for use in clinical studies as well as studies of therapeutic interventions. We realize that clinical studies can be expensive, and will consider clinical and translational proposals with budgets up to $350,000 per year. This grant mechanism is not intended to support large, randomized clinical trials of medications.

Reproducibility:

Proposals that focus on reproducing previous results are often at a disadvantage in review panels because, almost by definition, they don’t bring new ideas. But we are aware of the critical importance of replication, especially for potential clinical interventions. As an example, a few teams have shown that correcting a genetic lesion even in adulthood can reverse characteristic symptoms in various genetic mouse models of autism. These are potentially paradigm-shifting experiments, as they suggest that early neurodevelopmental defects may be repaired or compensated for later in life. But there is an urgent need for them to be replicated and extended. And if they cannot be replicated, the ‘negative’ findings need to be swiftly communicated to the research community. In short, we will consider requests that involve replicating and expanding key findings.

Data sharing:

We expect SFARI investigators to share renewable reagents and data developed using SFARI funds with other qualified investigators. In the case of several projects for which early release of data is critical, we made funding contingent on pre-publication sharing of results with other SFARI investigators. In these cases, we added important safeguards to ensure that the researchers are protected in various ways and can’t be scooped on their own data. In a Viewpoint published last October, Randy Buckner articulates the critical need to share data, and how best to balance this need with protecting the intellectual contributions of the investigators.

Returning investigators and percent effort:

We try to balance the needs of new investigators with continuation applications from current investigators. We have always been enthusiastic about bringing in investigators who have a strong track record in another research area and can offer a fresh approach to autism research.

Even more so than in the past, we will prioritize new grant applications and provide only limited funding for renewal of three-year Research Awards. After three years of support, grantees are encouraged to seek continued funding from the National Institutes of Health and other agencies; when justified by exceptional progress, we will consider limited transitional funding. We will continue to welcome former grantees’ attendance at SFARI-sponsored meetings, workshops and other activities. We do encourage recipients of SFARI Pilot or Explorer Awards who have achieved success in their initial studies to apply for three-year Research Awards.

We seek applications from independent investigators who can devote a substantial portion of time to their SFARI project. Although we do not require a minimum Principal Investigator effort, we expect the investigator to commit to a level of effort that reflects a leading role in the project.

A note about budgets:

Proposals are judged to some degree on whether the budgets are reasonable given the potential risks and rewards of the research (larger budgets raise the bar for funding and are subject to heightened scrutiny), and whether there is funding from other sources. At the full-application stage we are especially vigilant about how specific aims overlap with other current or pending grants. We also work with successful applicants to adjust budgets after the review process.

Our referees are a diverse group of scientists who dig into details of the proposals (the study plan, investigator’s track record, quality of preliminary data, data-sharing plan and budget). Some referees have been with us for many years, but each year we add and subtract a few members.

In the past few years, we’ve included some SFARI investigators. After an initial electronic review, in which referees submit short comments (proposals that do poorly in the electronic pre-review are rejected without full discussion), we hold a lively two-day meeting to discuss the grants in person. At that meeting, referees present grant proposals, and we together debate strengths and weaknesses and, eventually, rank each proposal. We discuss conceptually related proposals in groups so that we can make direct comparisons.

We don’t divide the committee into subgroups by expertise. All members are present for discussion of all proposals — except, of course, when there are potential conflicts of interest. This approach may be less efficient than that of a specialized subcommittee, but the upside is that it offers a check against overspecialization unduly influencing decisions.

The referees and SFARI staff both have substantial flexibility in suggesting changes to the scope or budget of a grant application. The feedback that we get from referees about this flexibility has been very positive compared with other types of review committees that are less interactive.

After the review committee meeting, we provide our Scientific Advisory Board with the opportunity to comment on tentative funding decisions. The board’s role is to help judge consistency and rigor of the review process, and to ensure a balance of the overall portfolio in line with SFARI’s mission.

Award notifications:

After the review committee and Scientific Advisory Board meetings, the Simons Foundation Board of Directors reviews the funding decisions. All awards require approval by this board. After this, we communicate the decisions to all applicants. We try to give investigators we do not fund a summary of the reasons for the decision. However, we ask applicants to keep in mind that referees are not asked to prepare formal remarks for the applicants, and it can be challenging to summarize the verbal discussion.

Explorer Awards:

In addition to our yearly RFA cycle described above, two years ago we launched the SFARI Explorer Awards. Unlike Pilot Awards granted through the annual RFA, applications for Explorer Awards are accepted throughout the year on a rolling basis, are for one year only and are capped at $50,000 in direct costs.

Explorer Awards are intended to provide resources to support exploratory experiments that strengthen hypotheses and lead to competitive applications for larger-scale funding from SFARI or other organizations. We review the applications internally with a response time as short as 30 days. What distinguishes both Explorer and Pilot Awards from Research Awards is that they place less emphasis on preliminary data. We receive five to ten Explorer applications per month, and have funded about 20 percent of them.

We always welcome feedback about our decision-making process. The more specific the feedback, the better. “I can’t believe you didn’t fund my grant; you guys make lousy decisions,” is less helpful than a reasoned critique of some of the criteria I have outlined above. We use your feedback to improve our process of review. You should not expect that we will change our decision on your grant as a result of your feedback.

Also, please keep in mind that a negative decision this round is not a broad judgment about the quality of work emerging from a lab, or a blanket statement about the potential for future SFARI funding. It’s simply that the abundance of high-quality grants and a finite budget force us to make tough decisions.