Tardelli Stekel PRO

Not with one account, as far as I can see. The owner and org members skip the gate, so my token never sees its 403.

Two things I can do alone:

1. A fresh public repo, gated with manual approval and nobody approved. The ~20/day that public repos pick up from outside would now land on the gate. If refusals count, it reads about what F did on its first day. If they don't, it stays near 0. That tests the anonymous refusal at the gate itself, at volume, not just p0's single 401 on a private repo. It's up: tardellirs/hub-download-count-gated-a, public and gated since 17:36 UTC today, no requests from us. Tomorrow's read covers about 14 hours, the 20th a full day.
2. A fine-grained token scoped to one repo, used on another private repo: logged in and refused. That's the token-scope check, though, not the gate, so it doesn't fill the 403 cell.

For the real gate 403, the cleanest is your account. I gate a fresh repo and don't approve you. You send one HEAD with your token at a time we agree, and nothing else touches it. Would you be up for that?

Yes, both are logged: the anonymous HEAD and the token HEAD each got a 302, the redirect to storage, not a 200. That's the same code the pulls that counted got: both s1 HEADs were 302s and s1 read 2. A denial looks different. An anonymous HEAD on a private repo (I just tried p0) gets a 401. So the anonymous pull saw the repo as public.

On a cached denial: /resolve/ doesn't look cached at the edge. Two HEADs in a row on a public LFS file both came back with cache-control: no-store and "Miss from cloudfront". So each request reaches the origin.

Side effect: p0 now has exactly one anonymous 401, from this afternoon (UTC). Tomorrow it also tells us whether a denied request counts.

On 288: yes, but per repo per address. A CI job pulling every 30 s across three repos reads up to 864.

Today's batch moved all four at once.

s1 (40 s apart, across 10:25:00) read 2, and s2 (4 min 30 s apart, inside 10:30–10:35) read 1. So it's 5-minute clock buckets, not a window anchored on the first pull.

t1 (two addresses in the same /64, 5 s apart) read 2, so the key takes the full address, not the prefix. u1 (one address, two tokens of the same account, 5 s apart) read 1, so a different token doesn't split the count. What's left is whether the account is in the key at all, or only the address.

On a fresh public repo: it won't stay quiet. F took no request from us and read 19 on its first day, G and H 17 each, so a +1 would disappear under that. Instead I made two private repos public for about 10 seconds this morning. v1 got one anonymous HEAD and one token HEAD 5 s later, from the same address and inside one bucket. v0 got the same flip and no requests, as a control for anything that reaches it in those seconds. If v0 reads 0, then v1 at 1 means the key is the address alone, and v1 at 2 means it also sees who is asking. Those read tomorrow.

q1's VPS pull went out over IPv6: the VPS resolves huggingface.co to its AAAA first, and the tunnel left from its v6 address. So q1 was already v6 to v6, across two networks. t1 is the first pair inside one /64.

Fair point on the token. I've just run u1 on a fresh private repo: two HEADs 5 s apart from one IPv6 address, each with a different read token on my account. It reads tomorrow with s1, s2 and t1. I have no second account, so token vs user stays open unless that one splits.

No need to touch the VPS. The Mac's home line already holds two addresses in one /64: the stable one and the temporary privacy one. Both work as a source, so I ran it just now on a fresh private repo, t1: one HEAD from each, same token, at 16:41:00 and 16:41:05 UTC, inside one 5-min bucket. It reads tomorrow with s1 and s2.

If t1 reads 2, one host counts once per address it holds. If it reads 1, HF merges on a prefix, and the next question is how wide. The VPS has a /48, so two addresses in different /64s under it would tell /64 from something wider.

Oct 16 results, read after today's batch:

• q0, untouched: 0
• q1, one HEAD from each of two IPs, same token, 5 s apart: 2
• q2, two HEADs from one IP, same token, 5 s apart: 1
• r3, two HEADs at 08:21:20 and 08:24:50: 1
• r1, at 08:21:20 and 08:25:20: 2
• r2, at 08:21:20 and 08:27:20: 2

So p4 = p6 merges on IP, not on the token: the same token from two IPs counted twice. (One pull went out over a home IPv6, the other from a VPS.)

10-min buckets are out: r1 would have read 1. 5-min clock buckets fit every repo so far, these and p0 to p7: 08:21:20 and 08:24:50 share the 08:20 bucket, and 08:25:20 is past the boundary. I was wrong that 4 and 6 min would settle it, though. An anchored window still fits if it's between 3:30 and 4:00. That's an odd length, but these pulls don't rule it out.

Two pairs would: 40 s apart across a :x5:00 boundary (buckets give 2, an anchored window gives 1), and 4:30 apart inside one bucket, :x0:10 and :x4:40 (buckets give 1, an anchored window under 4 min gives 2).

Exact times for p5's first three, logged when each download finished (about 2 s after its HEAD, the same lag for all three): 18:08:34, 18:11:06, 18:13:38 UTC. The script slept 150 s between pulls, so the gaps are 2:32 and 2:32, and the first to the third is 5:04. No 4:50 hiding in the rounding.

That rules out a window that restarts on every request, which would have given 6. It doesn't rule out 10-min buckets: the first pull fell at :08, inside your :x5 to :x9 range, so the 18:10 boundary splits it from the next two and gives 7 as well. An anchored window would be between 2:32 and 5:04. Pulls on fresh private repos at 4 min and at 6 min after a first one, started at :x1 or :x2 so that no :x0 boundary falls inside, would separate all three.

The floor is fading on our side too. From Oct 14 to 15, A to F moved +7, +8, +6, +11, +8 and +7: about 20 on day one, 11 on day two, 7 on day three.

Agreed on the two-IP test for p4 = p6. It's running this morning on fresh private repos: same token, one HEAD from each of two IPs 5 s apart, against two HEADs from one IP. The same batch carries the window test, with second pulls at 3.5, 4 and 6 min after a first one at 08:21:20. Numbers on Oct 17.

The private run is in, and it reads clean. Eight private datasets, all pulled with my token from one IP on Oct 14 between 18:08Z and 20:24Z, read after today's batch:

• p0, untouched: 0
• p1, one HEAD to /resolve/ (redirect not followed): 1
• p2, one xet-read-token call only: 0
• p3, one hf_hub_download: 1
• p4, eight different files within 47 s: 1
• p6, the same file eight times within 47 s: 1
• p7, two files 81 min apart: 2
• p5, eight files, gaps of 2.5, 2.5, 27, 22, 33, 23 and 25 min: 7

So the counted call is the one on /resolve/, and a HEAD alone is enough. The token call counts for nothing, so F's 19 came entirely from the floor. Authenticated pulls do count.

Requests merge per repo, not per file: p4 reads the same as p6. p5 says something about the window. The pull 2.5 min after the first merged, but the one 5 min after the first counted, though it came only 2.5 min after the one before it. That fits a window of about 5 min that starts at the counted request and doesn't slide. Fixed clock buckets of 5 or 10 min would also give 7, so this run can't tell those apart.

p0 answers your F2 question from the other side. An untouched private repo stays at 0, so the counter adds nothing on its own clock, and the floor on public repos is real traffic from whatever can see them. G (one HEAD) and H (one full download) both read 17 on their first batch, as expected under that floor.

To tell the anchored window from clock buckets, the next run would be fresh private repos with a second pull at 4 min and at 6 min after the first.

A correction first, and it changes the picture. I read F too early. The batch doesn't land all at once: A to E moved between 10:22Z and 10:37Z, F only between 11:25Z and 11:34Z, and F now reads 19. So two lines in my last reply are wrong: that the token call alone doesn't count, and that the floor never touched F. Sorry for that.

F took no /resolve/ request at all, only one xet-read-token call, and it reads 19, between A (17 after eight pulls) and D (21 after one HEAD) on their first batch. That answers the F2 question before F2 exists: a repo climbs with no request from us. The floor isn't switched on by a /resolve/ hit, and it doesn't wait for age either. F got its 19 inside its first day, like 99% of the datasets created that day.

So on a public repo, one counted request is +1 under about 20 on day one and about 11 a day after. G and H can't show the count or the switch. I'll still read them tomorrow, but I don't expect them to say more than F did.

What could still work is a repo nothing else can read: a private dataset. Private repos do keep a counter: mine show 927 all-time downloads across 56 of them, small numbers that fit my own use. With no crawler able to reach one, every count on it would be ours. The open question is whether an authenticated pull counts the same as an anonymous one, and the private run would answer that too.

Would a private run answer what you were after, or does the token change too much?

Today's batch landed at our 10:37Z tick, again cut at about 08:00Z.

A to E, which nobody pulled again: 28, 31, 28, 35, 31. That's +11, +11, +10, +14, +11 in a day. So the floor isn't a first-day burst. It keeps coming at about 11 a day per repo, and the 1,830 public datasets created on 10-12 moved the same way (median 31 to 46).

F reads 0. Of the 80 public datasets created in its hour (09Z on 10-13), it's the only one: the other 79 have counts, median 34. So two things at once: the token call alone doesn't count, and whatever feeds the floor never touched F.

The second part I can't explain yet. F has the same owner, layout and card as A to E. The one difference I can see: each of A to E took at least one request to /resolve/ (the GETs, or the HEADs on D and E), and F took none.

That makes F the first quiet repo we have, and suggests the next run: leave F alone as the control, and make two fresh repos, G with a single HEAD and H with a single hf_hub_download. If F stays at 0, G and H read clean against it. And if the floor only starts after a /resolve/ hit, G and H will keep climbing past 1 in the days after.

Thank you, and the credit is earned.

Counters still read 0 on all five, 17 hours after the pulls (last check 08:51 UTC). The poll keeps running every 15 minutes until Wednesday.

Your read of the Xet path matches the code. In huggingface_hub 2.2 with hf_xet, each hf_hub_download sends a HEAD to /resolve/, follows the xet-auth link to /api/datasets/{repo}/xet-read-token/main, and fetches the bytes from CAS. No GET reaches /resolve/.

F is up, and I didn't wait for D. hub-download-count-test-f got one xet-read-token call at 09:04:00 UTC today, from the same IP as A to D, with no HEAD and no CAS fetch. If D reads 0, F tells us whether the token call is the counted one. If D reads 1, F still tells us whether the token call counts on its own.

One correction on E: it isn't clean. At 16:33 UTC yesterday I sent a single HEAD to E's part-0.bin from a different IP, to check that the file sits on Xet. So E is a second D from another IP, not a baseline. The viewer never read any of the five: the datasets server reports no supported data files, so the offset you expected from it isn't there.

And you're right that A, B and C give the per-repo check for free: they started within a second of each other.

Agreed on E: it's the offset, not noise. The viewer did look at all five and found "No (supported) data files", since they're .bin, so I expect 0 there. E will show it.

One correction on A. hf_xet was installed, so for each file hf_hub_download sent a HEAD to /resolve/, then a call for a Xet read token, and fetched the bytes from the Xet CAS. No GET ever hit /resolve/. So A reads 1 if merged per repo and 8 if per file, with no 16. D is the HEAD alone, without the token call.

Timestamps, UTC: A 15:34:30 to 15:35:17. B 15:34:30 to 15:52:12, 2.5 minutes apart. C 15:34:31 and 15:46:32. D 15:34:28.

On the next run: yes, but bracketed instead of one gap. Counts are per repo, so I can run several pairs at once: two pulls 4, 6, 8 and 10 minutes apart, one repo each. C already caps it at 12. If C reads 1 too, it isn't a 5-minute window at all, and the gaps move to 30 minutes, 2 hours and a day.

Thanks for keeping your IPs off them. And thank you for all these questions: I published the Model Pulse write-up today and thanked you in it, since your questions fixed three data issues and led to the size analysis. Article: https://huggingface.co/blog/tardellirs/model-pulse

Good point on the clock: bytes are where it would hide, and files soaking most of it up points away from it.

Running it now. None of my datasets is quiet enough, so I made five empty ones (tardellirs/hub-download-count-test-a to -e) with 4 KB files. All pulls come from one IP, without a token, starting 15:34 UTC today:

• A: 8 files in under a minute, the control.
• B: 8 files, one every 2.5 minutes, 17.5 minutes in all.
• C: 2 files, 12 minutes apart.
• D: a single HEAD, no GET.
• E: never pulled, a baseline for crawlers.

Fixed 5-minute windows would give B about 4 and C 2. A window that extends while requests keep coming would give B 1 and C 2. I'll post the counts here once they update.