publications

2026

1. Preprint

   

   From Uncertain Judgments to Calibrated Rankings: Conformal Elo Estimation for LLM Evaluation

   Bora Kargi and David Salinas

   arXiv preprint arXiv:2606.13221, 2026

   Turns biased LLM-as-a-judge votes into calibrated model rankings with honest, distribution-free uncertainty bounds — at a fraction of the cost of human evaluation.

   Abs arXiv Bib PDF Code

   LLM-as-a-judge evaluation is cheap but biased, which can badly miscalibrate model rankings. We address this at two levels: locally, by propagating calibrated win probabilities (instead of hard labels) into the Bradley–Terry procedure, bringing LLM-derived Elo ratings within 17.9 MAE of human ratings on LMArena; and globally, by using split conformal prediction to place distribution-free uncertainty intervals around the LLM-vs-human rating gap. The result is a low-cost evaluation tool that gives developers calibrated Elo estimates with honest uncertainty bounds, without large-scale human annotation.

   @article{kargi2026softelo,
     title = {From Uncertain Judgments to Calibrated Rankings: Conformal Elo Estimation for LLM Evaluation},
     author = {Kargi, Bora and Salinas, David},
     journal = {arXiv preprint arXiv:2606.13221},
     year = {2026},
     note = {Turns biased LLM-as-a-judge votes into calibrated model rankings with honest, distribution-free uncertainty bounds — at a fraction of the cost of human evaluation.},
   }

2. Preprint

   

   Half-Truths Break Similarity-Based Retrieval

   Bora Kargi, Arnas Uselis, and Seong Joon Oh

   arXiv preprint arXiv:2602.23906, 2026

   CLIP-style models often score a caption as more similar after a plausible but wrong detail is added; we fix this by supervising the individual entities and relations within each caption.

   Abs arXiv Bib PDF Code

   CLIP-style image-text models can be fooled by "half-truths": appending a plausible but incorrect detail to an otherwise correct caption often increases the similarity score instead of lowering it. We introduce CS-CLIP (Component-Supervised CLIP), which decomposes captions into entity and relation units and trains the model to score each correct unit above a minimally-edited incorrect foil, while preserving standard dual-encoder inference. CS-CLIP raises half-truth accuracy from 40.6% to 69.3% and improves average performance on established compositional benchmarks by 5.7 points.

   @article{kargi2026halftruths,
     title = {Half-Truths Break Similarity-Based Retrieval},
     author = {Kargi, Bora and Uselis, Arnas and Oh, Seong Joon},
     journal = {arXiv preprint arXiv:2602.23906},
     year = {2026},
     note = {CLIP-style models often score a caption as more similar after a plausible but wrong detail is added; we fix this by supervising the individual entities and relations within each caption.},
   }

2025

1. BMVC 2025

   

   Evaluating Self-Supervised Learning in Medical Imaging: A Systematic Investigation of Robustness, Generalizability, and Multi-Domain Transfer

   Bora Kargi, Valay Bundele, Karahan Sarıtaş, Oğuz Ata Çal, Kıvanç Tezören, Zohreh Ghaderi, and Hendrik Lensch

   In British Machine Vision Conference (BMVC), 2025

   A systematic benchmark of self-supervised learning across 8 methods and 11 medical-imaging datasets, measuring label efficiency, cross-dataset generalization, and out-of-distribution robustness.

   Abs arXiv Bib PDF

   Self-supervised learning (SSL) is promising for medical imaging, where labelled data is scarce, but prior studies evaluate it only in narrow settings. We run a systematic study of 8 major SSL methods across 11 real-world medical datasets, measuring in-domain performance at 1%, 10%, and 100% label fractions, cross-dataset generalization, and out-of-distribution robustness. We further analyze how initialization strategy, architecture (ResNet-50 vs. ViT-Small), and multi-domain pretraining shape SSL performance, giving practical guidance for building robust, generalizable, and clinically viable models.

   @inproceedings{bundele2025ssl,
     title = {Evaluating Self-Supervised Learning in Medical Imaging: A Systematic Investigation of Robustness, Generalizability, and Multi-Domain Transfer},
     author = {Kargi, Bora and Bundele, Valay and Sarıtaş, Karahan and Çal, Oğuz Ata and Tezören, Kıvanç and Ghaderi, Zohreh and Lensch, Hendrik},
     booktitle = {British Machine Vision Conference (BMVC)},
     year = {2025},
     note = {A systematic benchmark of self-supervised learning across 8 methods and 11 medical-imaging datasets, measuring label efficiency, cross-dataset generalization, and out-of-distribution robustness.},
   }

2. ACL 2025

   

   Scholar Inbox: Personalized Paper Recommendations for Scientists

   Markus Flicke, Glenn Angrabeit, Madhav Iyengar, Vitalii Protsenko, Illia Shakun, Jovan Cicvaric, Bora Kargi, Haoyu He, Lukas Schuler, Lewin Scholz, Kavyanjali Agnihotri, Yong Cao, and Andreas Geiger

   In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), 2025

   An open-access platform offering personalized paper recommendations, visual summaries, and a map of science, released together with a dataset of 800k user ratings.

   Abs Bib HTML PDF Website

   Scholar Inbox is an open-access platform that helps researchers keep up with a rapidly growing literature through personalized paper recommendations, continuous updates from open-access archives (arXiv, bioRxiv, etc.), visual paper summaries, and semantic search. A recommendation model trained on user ratings tailors a daily feed to each researcher, while a "map of science" addresses the cold-start problem and enables cross-domain exploration. We release a dataset of 800k user ratings and evaluate the system both quantitatively and through an extensive user study.

   @inproceedings{flicke2025scholarinbox,
     title = {Scholar Inbox: Personalized Paper Recommendations for Scientists},
     author = {Flicke, Markus and Angrabeit, Glenn and Iyengar, Madhav and Protsenko, Vitalii and Shakun, Illia and Cicvaric, Jovan and Kargi, Bora and He, Haoyu and Schuler, Lukas and Scholz, Lewin and Agnihotri, Kavyanjali and Cao, Yong and Geiger, Andreas},
     booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)},
     year = {2025},
     note = {An open-access platform offering personalized paper recommendations, visual summaries, and a map of science, released together with a dataset of 800k user ratings.},
   }