Comparative Evaluation of Publicly Accessible LLM APIs for Technical Prompt Response: A Case Study Using Gemini, OpenRouter, and GroqCloud

Q: How many days will it take for my paper to be published?

The review time for papers is not fixed. However, if the paper is accepted and the author completes the processing charges formalities, the paper will be published within a few working days.

Q: I would like to receive a hard copy of the journal materials. Are there any additional charges?

You can log in to the author portal and pay 500 INR to receive the hard copy materials.

Bharath Gowda M R; Abhilash S; B U Bharath; Prajwal K

Comparative Evaluation of Publicly Accessible LLM APIs for Technical Prompt Response: A Case Study Using Gemini, OpenRouter, and GroqCloud

Authors: Bharath Gowda M R, Abhilash S, B U Bharath, Prajwal K

Unique Paper ID: 176294
Volume: 11
Issue: 11
PageNo: 6339-6344

Keywords: No Keywords Found

Abstract:
The rapid evolution of large language models (LLMs) has made high-performance natural language understanding and generation accessible through public APIs. However, developers and researchers often face challenges in determining which LLMs provide the most relevant, accurate, and useful outputs for domain-specific tasks such as technical queries and coding assistance. This paper presents a comparative evaluation of three widely available LLMs—Gemini 1.5 Flash (via Google AI Studio), OpenRouter (Nemotron-49B), and GroqCloud (Llama 4 Scout 17B)—using a custom-built Streamlit interface. The system prompts all three models simultaneously and displays their responses side by side, followed by an automatic evaluation using GroqCloud(deepseek-r1-distill-llama-70b)", which scores the outputs on key metrics: Relevance, Technical Accuracy, Clarity, Code Correctness, and Usefulness. The responses are assessed based on prompts in the tech and programming domains. Results are visualized both in tabular and graphical forms, revealing notable performance variations across the models. Gemini and OpenRouter consistently achieved top scores across all metrics, while GroqCloud demonstrated slightly lower performance in clarity and technical precision. This study provides actionable insights for developers seeking the most effective LLM for real-world technical applications and lays a foundation for automated LLM evaluation pipelines using multi-model comparisons.

Download article

email to a friend

Cite This Article

ISSN: 2349-6002
Volume: 11
Issue: 11
PageNo: 6339-6344

Comparative Evaluation of Publicly Accessible LLM APIs for Technical Prompt Response: A Case Study Using Gemini, OpenRouter, and GroqCloud

Available:https://ijirt.org/article?manuscript=176294

21 Mar, 2025

SMART COLLAR TO MONITOR COW’S HEALTH

Impact Factor
8.01 (Year 2024)

UGC Approved
Journal no 47859

Join Our IPN

IJIRT Partner Network

Submit your research paper and those of your network (friends, colleagues, or peers) through your IPN account, and receive 800 INR for each paper that gets published.

Join Now