{"doi":"10.1093/reseval/rvw002","title":"Funding decision-making systems: An empirical comparison of continuous and dichotomous approaches based on psychometric theory","abstract":"<p>Psychometrics questions the use of dichotomous decisions. For these reasons, De Los Reyes and Wang (2012) favour a continuous funding decision system, in which the funded percentage of a requested grant sum is coupled to the ratings that a proposal receives in the ex ante peer evaluation. In contrast to the ‘winner takes all’ philosophy in a dichotomous funding decision system, a continuous system takes the low reliability of peer review ratings into account. Funding decisions are mostly based on peer review rating systems that have rather low inter-rater reliability. This article aims to use psychometrics to simulate the two funding decision systems, to compare them to the funding decision system implemented by a real funding organization, and with this, to investigate for the first time the effects of measurement errors on funding decisions. We used peer review data from the Austrian Science Fund (FWF) (N = 8,496 proposals), which obviously implements a hybrid funding decision system. The approval rate at FWF is 44.5%; our findings show that the approval rate would be 32.1% using a purely dichotomous system and 58.4% using a continuous funding decision system. As the funded percentage of a proposal’s requested grant sum increases with increasing mean ex ante peer evaluation of a proposal (<em>r</em> = 0.23), the FWF also shows elements of a continuous funding decision system. Relative to a continuous system, a dichotomous system reduces the approval probability of a proposal overall. This is even the case for high-quality proposals (approval probability ∼0.70).</p>","journal":"Research Evaluation","year":2016,"id":3061,"datarank":0.35910270494235336,"base_score":1.791759469228055,"endowment":1.791759469228055,"self_citation_contribution":0.26876392038420827,"citation_network_contribution":0.09033878455814508,"self_endowment_contribution":0.26876392038420827,"citer_contribution":0.09033878455814508,"corpus_percentile":null,"corpus_rank":null,"citation_count":5,"citer_count":3,"citers_with_citation_signal":3,"citers_with_endowment":3,"datacite_reuse_total":0,"is_dataset":false,"is_dataset_confidence":0.0557,"is_oa":true,"file_count":1,"downloads":112,"has_version_chain":false,"published_date":"2016-04-01","fair_score":null,"fair_percentile":null,"algorithm_id":"datarank_citation_only_1hop_v6","ranking_scope":"data_only","authors":[{"id":661,"name":"Lutz Bornmann","orcid":"0000-0003-0810-7091","position":1,"is_corresponding":false},{"id":5654,"name":"Hans‐Dieter Daniel","orcid":"0000-0001-7925-1846","position":3,"is_corresponding":false},{"id":12196,"name":"Rüdiger Mutz","orcid":"0000-0003-3345-6090","position":0,"is_corresponding":true}],"reference_count":38,"raw_metadata":{"citation_network_status":"fetched"},"created_at":"2026-03-01T18:20:47.508186Z","pmid":null,"pmcid":null,"fwci":null,"citation_percentile":null,"influential_citations":0,"oa_status":null,"license":null,"views":0,"total_file_size_bytes":0,"version_count":0,"fair_f":null,"fair_a":null,"fair_i":null,"fair_r":null,"fair_zscore":null,"fair_rationale":null,"fair_model":null,"fair_agent_version":null,"fair_fulltext_source":null,"fair_has_llm":null,"fair_computed_at":null,"clinical_trials":[],"software_tools":[],"db_accessions":[],"linked_datasets":[],"topics":[]}