Privacy policy

What we collect

• Your account. You sign in with GitHub or Hugging Face. We receive and store your name, email address, profile picture URL and handle (used as your default username), plus the sign-in tokens the provider returns. We do not receive your GitHub or Hugging Face password.
• Speed tests. The model, inference engine and settings, measured results (tokens per second, time to first token, memory use), and your hardware: GPU or chip, count and memory, CPU, RAM, operating system, and optionally power draw and what you paid for each component. Runs can also carry evidence captured by the measuring client (such as the CLI): hashes of the prompt and output, short excerpts of both, and the engine's own timing data. Any notes you add are stored too.
• Evals and other content. Eval runs (including per-question prompts, responses and scores), eval suites and datasets you register, model reports and their images, comments, reactions and ratings, training projects, marketplace listings and photos, and rental listings.
• Messages. Marketplace messages are stored so the sender and recipient can read them.
• API keys. When you create a key (in the dashboard or through the CLI sign-in), it is shown to you once. We keep only a SHA-256 hash of it, its first 12 characters for display, and when it was last used.
• Payments. Your Stripe customer ID, subscription status and renewal date, credit balance and credit history, rental usage (token counts and credits charged), and, if you receive rental earnings, your Stripe Connect account ID and payout records. Card details are entered on Stripe's checkout pages and never reach our servers.
• Ad clicks. When you click an ad, we record which ad, the time, the referring page and your browser's user agent. To ignore repeat clicks we also store a keyed hash of your IP address and user agent that changes every 10 minutes. We do not store your IP address itself, and clicks are not linked to your account.
• Logs and errors. Our servers log API requests (route, status, timing and, for some actions, your user ID). Credentials and cookies are redacted from these logs. If something breaks, technical details about the error may be sent to Sentry.

How we use it

• To run the site: show your submissions, rank them on leaderboards, and let you manage your content.
• To check results. Each speed test is checked automatically (canonical prompt, evidence, batch size, and a physical speed ceiling). Results far above what the hardware could plausibly do are marked suspicious and hidden from leaderboards. Accounts with three or more verified runs are marked verified.
• To enforce submission limits and the limits of your plan.
• To process Pro subscriptions, credit purchases, rental billing and owner payouts.
• To count ad clicks.
• To find and fix bugs.

What is public

Anyone, signed in or not, can see the following, on the site and through the public API:

• Your profile: username, profile picture, join date, verified status, whether you are on Pro, and your approved runs. Your display name appears next to things you post. Your email address is never shown.
• Approved speed tests with their hardware details, purchase prices, notes and evidence excerpts. Runs marked suspicious are hidden from leaderboards but listed on the suspicious runs page.
• Approved eval runs, including the per-question prompts and responses that were submitted with them.
• Published reports, their images and comments. Unpublished report drafts are visible only to you and site admins.
• Marketplace listings, including the photos and the contact method you write into the listing. Listings are also available as a public feed.
• Active rental listings. The endpoint URL and key you give us are encrypted (AES-256-GCM) and never shown to other users.

Public pages can be indexed by search engines and shared with link previews.

Cookies and local storage

We do not use third-party analytics or advertising cookies. The site sets:

• A session cookie that keeps you signed in, and short-lived cookies used during sign-in (15 minutes) and for request protection.
• A language cookie (NEXT_LOCALE) that remembers the language you picked.
• A theme cookie and local storage entry for light or dark mode (cookie lasts one year).
• Local storage entries for dismissing the sign-in prompt (for 7 days) and for the runs you picked to compare.
• A session storage entry that carries a just-submitted run over to your dashboard.

Services we rely on

• GitHub and Hugging Face for sign-in.
• Hugging Face API for public model information. No data about you is sent.
• Stripe for subscriptions, credit purchases and payouts to rental owners.
• Sentry for error reports.
• Cloudflare, which sits in front of the site and handles incoming requests.
• Hetzner for our servers. Uploaded images and eval files are kept in S3-compatible object storage.
• Rental owners. When you use a rental, your requests are forwarded to the owner's own server. We record token counts and credits charged, not the content of your requests or responses, but the owner's server does receive that content.

Payments

Pro subscriptions and credit packs are paid through Stripe Checkout, and you manage or cancel Pro through Stripe's billing portal from the Pro page. Rental owners who want payouts connect a Stripe account; Stripe handles their identity and bank details.

Your choices

• Delete your speed tests, saved setups and marketplace messages from your dashboard.
• Delete your eval runs from the eval pages, and delete or unpublish your reports.
• Edit or delete your marketplace listings and their photos.
• Revoke API keys at any time from your dashboard.
• Turn direct messages on or off (they are off by default), and change your username once.
• Cancel Pro from the billing portal.
• There is no self-service account deletion yet. To delete your account or anything you can't remove yourself, contact us as described below.

Contact

Open an issue on our GitHub repository (issues are public, so don't include private details there) or message @lottolabs on X. See also our Terms of use.