"Within light there is darkness, but do not try to understand that darkness;
Within darkness there is light, but do not look for that light." -- Sandokai
Being an academic for almost 5 years now, one of the most fascinating things about academia for me is how rarely people talk about the rejections and failures they faced in their careers. Paradoxically, almost everyone says they faced rejections at least as frequently (if not more) as acceptances on papers, proposals, and honors/awards. And yet, most academic CVs have pages upon pages of publications, grants, and honors/awards and not a word on the rejections. I fear this status quo may be perpetuating survivorship bias and/or impostor syndrome, in turn exacerbating mental health issues endemic in academia and especially among people from underrepresented and/or marginalized groups. I have been speaking about these issues at UCSD, e.g., this oSTEM panel discussion. I hope this post, and my new CV addendum described below, is a step toward changing this status quo.
In my opinion, acceptances and rejections are two sides of the same coin of intellectual progress even though the latter are clearly painful. As long as I have the confidence that an evaluation process is likely fair and scrupulous, I take rejections in my stride (but if it is not, I do not sit silently either!). Likewise, competition is an inevitable reality in all walks of life. Interestingly, "curriculum vitae" literally means "course of life" in Latin. To me, success and failure are both integral parts of life and equanimity is crucial for self-growth. So, why then do academics stay silent on rejections in their CVs? How do we find out the reason(s)? Run a basic Twitter poll, of course! :) About 5 in 7 picked "waste of time/space" as their reason. The rest picked "embarassment", "fear", or "something else" but I did not see any specific reasons outlined for such fears or otherwise. The "waste" interpretation is likely rooted in academic incentive structures explicitly valuing only acceptances and ignoring rejections.
Of course, detailed lists of paper rejections may be boring. But summary statistics on rejections are certainly interesting, feasible, and IMO appropriate to share in a CV, the "living" documentation of one's professional life. Thankfully, I already maintain spreadsheets with details on my paper submissions and proposal/award applications. With simple spreadsheet formulae, I created the following summary. I have added them as an addendum in my CV. I will update these periodically, just like the rest of my CV. I hope this data "schema" is helpful for more people considering starting such a practice.
NB: For the sake of simplicity, "paper" below includes all forms of peer-reviewed publications: full research papers at conferences, demo papers, workshop papers, posters/abstracts, and journal papers. An (eventually) accepted paper or awarded proposal will be counted more than once across these statistics if it was not accepted or awarded directly in the first attempt. I will also note that all but one of my accepted VLDB / SIGMOD papers went through a revision decision.
Summary Statistics on Paper Decisions
Summary Statistics on Proposal (Grant / Gift) Decisions
Rejections on Major Competitive Honors or Awards
2021:
Nonprofit: Sloan Research Fellowship
Industry: Google Research Scholar Award, Microsoft Research Faculty Fellowship, VMware Systems Research Award
2020:
Industry: Amazon Research Award, Sony Focused Research Award
2019:
Federal: NSF CAREER Award
Industry: Bloomberg Data Science Research Grant, Google Faculty Research Award, Microsoft Investigator Fellowship, Microsoft Research Faculty Fellowship, NetApp Faculty Fellowship
2018:
Federal: NSF CRII Award
Industry: Amazon Research Award, Bloomberg Data Science Research Grant, Facebook Research Award
2017:
Federal: NSF CAREER Award
Some Salient Aspects from the Above Summaries
To be honest, I did not expect to glean much from this exercise going in, except perhaps for "Ugh, why am I re-living all these rejections?!" :-/ Instead, many salient aspects stood out for me. Make of these what you want.
1) I was pleasantly surprised to see paper Rejects were not as dominant as I thought, although they were not far behind Accepts each year. Even if Rejects were dominant, I think seeing Accepts alongside like this will help me retain a sense of proportion.
2) Related to the above, it appears Revise decisions at VLDB / SIGMOD consistently played a positive role in mitigating the stark Accept-Reject dichotomy. As I said earlier, all but one of my accepts at VLDB/SIGMOD came after a revision. In fact, Sam Madden had highlighted this point in his comment on my recent blog post on the DB venues. So, yeah, I empathize with my non-DB area CS peers still fighting for their "Right to Revision", e.g., see this blog post. ¡Viva la Revisión! :)
3) Given the above, it is not surprising to me that my students (and I) preferred/prefer to submit our research to VLDB / SIGMOD. That said, we did dip our toes into many nearby venues--SysML/MLSys, KDD, ICML, and SOSP--albeit with no success at any of them so far. :-/
4) I'll admit it did put a smile on my face to see that two-thirds of my papers got accepted in the first attempt itself. :) I did not tally full research papers separately in that histogram but even so that bar remains the largest. At the other extreme, 2 papers faced Reject 3 times and 1 paper faced it 4 times (!) before an eventual Accept--all still at the "top tier" (VLDB / SIGMOD). This was only possible due to the amazing perseverance of my students. While I knew that persistence is crucial, I am thankful to my students for inspiring and reinforcing my own resolve through these rejections.
5) My proposal rejection statistics are clearly worse. But many senior faculty have told me it does not get better. :-/ Proposal acceptance rates in the US are a function of the (im)balance of competition vs research budgets. Interestingly, my submission counts to industry are higher than federal agencies although both numbers are high. I faced similar rejection rates at both of these types of sources.
6) My long list of award/honor rejections looks sad indeed. Then again, I went after most major ones, and more volume begets more rejections (and acceptances). In hindsight, I am not sure if pursuing all of them was prudent. For instance, Sloan and MSR demand bespoke summary proposals that require non-trivial time to craft. And one needs senior nominators and letter writers. Was all that time/effort (mine, nominator's, and letter writers') worth it given the very high rejection rates? I do not know. :-/
7) Finally, if you are wondering which of these gazillion rejections was the most painful for me, I do have an answer: NSF CAREER Award. I tweeted all about it: first rejection, second rejection, and third attempt being funded. I cannot but wonder if "pity" played a role at NSF in the end but hey, it is funding all the same. I accepted all outcomes with humility, and I grew from all experiences. This one did lead me to learn a lot about how NSF rolls. Later on, NSF course-corrected on its CAREER criteria and I tweeted about that too, expressing my joy on how it will help US academia.
Anyway, I hope the above conversation on rejections, my summary statistics, and the highlights I spoke about were interesting and/or useful to you. I recognize some of it may come across as "humble brag" but hey, I will take humble bragging any day vs moping about my rejections. :) Feel free to let me know in the comments section if you have any thoughts/questions on all the above or have your own experiences to share on this whole topic.
