Cosmic Bull

Rendered from benchmarks/README.md at commit f191063fd89d in the project repository. The committed file is the source of truth; this page is a rendering of it.

Benchmarks

What has been benchmarked, and what each benchmark actually proved.

A benchmark here is not a performance measurement. It is a capability proof: an application built to test whether the agent can do a class of work end to end, with the result verified on a live chain rather than asserted.

Underlying transaction-level evidence: pearl/DEPLOYMENT.md.

1. Value handling — vault (§1)

Question: can the agent build a realm that takes custody of real coins and never loses track of them?

Result: PASS. 14 live transactions. The conservation identity held == UsersTotal + FeesAccrued + surplus was verified against real auth/accounts bank state after deposits, a fee change, a claim, an out-of-band send, and the final drain — exact equality at every checkpoint, final state all zeros.

What it taught (chain semantics absent from the local harness):

Two throwaway realms (cointest, cointest2) were deployed specifically to attack the vault from outside — forwarding ugnot out-of-band, and minting a junk denom into it. Both were swept correctly.

2. Composability — bounties (§2)

Question: can a second realm consume an already-deployed primitive without redeploying it?

Result: PASS. bounties imported the live p/…/feeledger — no dependency redeploy. Composed invariant held == bounties + users + fees + surplus verified exact at five checkpoints; final state all zeros with both bounty records terminal in Render.

The economically important finding — a fee-snapshot front-run defence: a bounty was created at 250 bps, the admin then set the fee to 0 in an attempted "waiver", and the award still charged the snapshotted 250 bps. Fees are priced at creation, not at settlement.

What it taught:

3. Primitive extraction — coinio + coindemo (§4)

Question: after the same coin-handling guards appear in several realms, can they be extracted into a reusable /p/ primitive without weakening them?

Result: PASS. coinio deployed and consumed by a fresh realm that resolves both on-chain dependencies (coinio + feeledger). H == U + S exact; final state all zeros; live Render reporting "conservation: OK".

Access control proven live: a non-owner Sweep was rejected at simulation ("owner only"), and ownership round-tripped to a second key and back.

This is the benchmark that justifies coinio's existence — a primitive that is written once and never reused is a cost, not an asset.

4. Multi-party application — grants (§5)

Question: can the agent build an application with distinct adversarial roles — creator, applicant, winner, expirer — and keep the authority boundaries intact under live attack?

Result: PASS. Conservation H == G + U + F + S exact at four checkpoints; final state all zeros; live Render showing both terminal records and "conservation: OK".

Live rejections, each an authority boundary:

The fee-snapshot defence held again: fee set to 0 after creation, award still charged the snapshotted 250 bps.

Permissionless expiry with correct beneficiary: hunter (not the creator) called ExpireGrant after the deadline, and the refund went to the creator — the expirer got nothing. Permissionless liveness without a value leak to whoever triggers it.

5. Multi-domain application — market (§6)

Question: can the agent build a realm composing four on-chain dependencies, with a two-step admin handover, and survive a broad attack battery?

Result: PASS. market resolves coinio + feeledger + avl + sanitize/v0 live. H == U + F + S exact at the sale (bank exactly 500,000 == 487,500 + 12,500) and at the final drain (bank "0", all views zero).

Seven distinct attack shapes rejected live at simulation (no gas wasted, no state touched):

AttackRejection
listing with a fee ceiling below the current fee"current fee 250 bps exceeds the seller's maximum 100"
buy with the wrong amount"send exactly the listed price: 500000ugnot"
re-buy an already-sold listing"listing is not active" — no double purchase
cancel by a non-seller"only the seller may cancel"
buy a cancelled listing"listing is not active" — buyer coins fail closed
AcceptAdmin by the wrong key"caller is not the staged admin"
out-of-band coin sendswept exactly, liabilities untouched

Two-step admin transfer round-tripped through a second key and back, with the wrong-acceptor rejected in between.

6. The GitHub → Pearl pipeline (§7–§11)

Question: can the agent take an application it did not design — existing code in an existing repository — and carry it to a verified live deployment without redesigning it?

Result: PASS, five times.

#ApplicationFindings resolvedLive attack shapes rejectedDeployed bytes
1fee_split0 RED / 3 YELLOW6T1+T2, occurrence-pinned
2timelock_guardian6 fixed in round 1; final 0R/0Y8byte-identical to commit
3upgrade_registry0 RED / 5 YELLOW8byte-identical
4permission_registry1 RED / 6 YELLOW7byte-identical
5service_registry1 RED / 7 YELLOW9byte-identical

What the pipeline proved:

Cost model established: a ~20 KB realm costs ~101k ugnot in gas fee plus a ~3.05M ugnot refundable storage-deposit escrow. The escrow dominates, and it can leave the signing key unable to fund the very transactions meant to verify the deployment. Adversarial probes run at simulate and cost nothing — so a constrained budget is never a reason to skip the attack battery.

7. The application factory — bounty_panel (§12)

Question: can the agent start from a bare objective — no repository, no named application — and carry it through discovery, architecture, reuse analysis, implementation, audit, deployment and independent live verification, making sound architectural decisions on the way?

Objective given: public bounties with an escrowed reward, contributor submissions, a designated resolution process to pick a winner, and on-chain payout to that winner — preventing unauthorized claims, double payouts and stuck funds.

Result: PASS. Live at r/g1ut6usp…/bounty_panel, tx 2c2a8dc8, height 608171, deployed bytes byte-identical to GitHub commit 62c5c42.

The architectural decision is the benchmark, and it was a subtraction. Discovery found the already-live grants realm satisfies 6 of the 7 clauses of the objective. The single genuine delta is that in grants the funder decides the winner, and the objective calls for a designated resolution process. So the work was scoped to exactly that delta — resolution authority separated from funding authority, fixed at creation, immutable after, exercised by an M-of-N panel voting over competing submissions.

What that bought:

New value-critical mechanisms (the parts no dependency provided):

Verification. 27 committed tests; two fresh-context adversarial audits → 0 RED, 3 YELLOW all addressed. Live: 17 transactions across 5 identities, three bounties driven to all three terminal states (awarded / expired / cancelled), and 21 adversarial calls each rejected with the intended guard. The conservation identity H == B + U + F + S held at every checkpoint, and the realm bank returned to exactly 0ugnot after 1,150,000ugnot round-tripped — the stuck-funds clause of the objective, demonstrated rather than asserted.

Cost, measured not extrapolated: 32,914 stored bytes → 163,078ugnot gas fee plus 4,882,000ugnot refundable storage deposit (≈148 ugnot/byte). A prior extrapolation of ≈5.05M happened to land near the total rather than the deposit — right number, wrong quantity, which is why the record replaced it with a measurement.

What it taught:

8. The application factory, second run — duebook + duebook_demo (§13)

Question: §7 asked whether the agent could go from a bare objective to a verified deployment. It could — but its headline decision was to reuse and subtract, and the open question left on the record was explicitly this: does the agent reach for composition even when composition is wrong, and build the primitive when building is right? This benchmark was chosen to test that.

Objective given: a reusable on-chain capability for Gno applications that need to schedule actions for future execution, enforcing deterministic execution rules, preventing replay, and allowing defined cancellation or expiration — useful to multiple independent applications, not designed around one. Explicitly: do not assume a new package is required, do not assume an existing component suffices, do not force the result into /p/ or /r/, and make the smallest architectural intervention the evidence justifies.

Result: PASS. Live at p/g1ut6usp…/duebook (height 609571) and r/g1ut6usp…/duebook_demo (height 609644), both byte-identical to GitHub commit 64cadd9.

The decision went the other way, and that is the finding

Discovery found scheduling machinery in the ecosystem and in Cosmic Bull's own live namespace — timelock_guardian most directly, plus governance-timelock patterns in the wider Gno ecosystem. Every implementation found was classified, and the classification is what drove the design:

Every scheduling implementation found either performs no effect, or performs one only as the tail of a governance vote.

That is a real gap, not a manufactured one. timelock_guardian schedules and gates, but the thing it gates is its own operation; it is not a component a second application can embed. The governance timelocks bind delay to a proposal lifecycle. Neither offers "authorize now, let an arbitrary consumer act later, exactly once" as a reusable part.

So unlike §7, the answer here was build, and the benchmark's value is that the two runs disagree. The factory is not biased toward reuse; it is biased toward evidence. Per the discovery rule, the claim on record is the bounded one — "No relevant existing implementation was found in the searched sources" — never ecosystem-wide uniqueness from a finite search.

The architecture — one idea, and a deliberate refusal

Gno has no autonomous execution. No cron, no keepers. "Scheduled" always means someone sends a transaction later. The naive design — store a callback and run it — is not merely unavailable, it is the wrong shape: it would mean a /p/ package holding code to execute on a consumer's behalf.

duebook refuses that. What a realm actually needs is not execution but authorization of a later transaction, exactly once:

d, err := book.Claim(id, now)   // checks due / not expired / still open,
if err != nil { panic(err) }    // and CONSUMES it before returning
// ... the realm performs its own effect here, under its own authority

Claim consumes then returns; the consumer acts afterward. Because the realm performs the effect itself, no closure, callback, or capability ever crosses a realm boundary — duebook cannot be handed code to run, so it cannot be tricked into running the wrong code. That is the security argument for the whole design, and it is an argument for doing less.

Replay is not guarded against, it is structurally impossible: IDs are allocated monotonically and never reused for the life of the Book, and a successful Claim removes the deferral before returning. At most one Claim per ID can ever succeed, whether the second attempt arrives in the same transaction or a year later.

The primitive holds no coins, performs no effects, imports no chain APIs, and owns no package-level state. The consumer supplies the clock. That last point is the sharpest edge and is documented as consumer contract item 1 — a realm that lets a caller choose now has no delay at all.

What it cost in new code

One new /p/ primitive, one reference realm. 579 deployed code lines.

BasisNewDependencyNew share
whole imported packages57996637.5%
reached dependency surface43739352.7%

Both are reported because they differ by 15 points and either alone misleads. Dependencies are p/nt/avl/v0 and p/nt/markdown/sanitize/v0; the primitive itself pulls in only errors, strconv and avl. 911 further lines of test were written and are not deployed.

Verification

46 committed tests (27 primitive + 19 realm), all passing against the chain-matched toolchain and re-run from a fresh clone before deploy.

(Correction, same day: this line first read "49 (30 primitive + 19 realm)". The realm figure was right and the primitive figure was not — grep -c '^func Test' pearl/p/duebook/duebook_test.gno returns 27, and a post-deployment re-run reports 27 PASS / 0 FAIL. The miscount is recorded rather than quietly overwritten: a verification section that inflates its own evidence is the exact failure this file exists to catch.) Adversarial audit: 0 RED, 3 YELLOW, all three remediated —

Live: 10 broadcast transactions across 2 identities, 7 adversarial refusals. Four deferrals driven to all four terminal outcomes — published, cancelled, expired-and-cleared, and published-by-a-stranger.

The central result. Deferral 1 was published at height 609664 and the identical call in a later transaction was refused with duebook: no such open deferral. Final state: OpenCount() 0, NextID() 5 — storage back to zero while the counter stayed advanced. That is the design's whole thesis made observable: replay is prevented by the never-reused counter, not by retained state, so bounded storage and permanent replay-safety are not in tension.

Exactly-once is a property of the deferral, not the caller. Deferral 4 was scheduled by one key and published by a different key, and the original scheduler's later attempt on their own deferral was refused identically. Publish is deliberately permissionless precisely so that nothing but Claim protects it — an access-control list guarding Publish would have made the experiment prove nothing.

What it taught

9. The application factory, third run — permbook + permbook_demo (§14)

Question: §7 chose reuse and subtract; §13 chose build a new primitive. Both were defensible, but both left the sharpest test unrun, and §13's own limitations section named it: would the agent correctly decline to build when an adequate component already exists? The false-negative direction. This objective was chosen because Cosmic Bull already owns a live realm that does exactly what was asked.

Objective given: a reusable Gno capability letting an application define named permissions, grant them to addresses, revoke them, and efficiently determine whether an address currently holds a given permission — suitable for composition by multiple independent applications. Explicitly: complete architectural freedom; do not assume a new implementation is necessary; do not assume an existing one is sufficient; do not assume the answer is a /p/ or an /r/; do not assume the first implementation found is the right one; the discovery result must be allowed to change the implementation decision.

Result: PASS. Live at p/g1ut6usp…/permbook (height 612427) and r/g1ut6usp…/permbook_demo (height 612731), both byte-identical to GitHub commit 319d405.

The test the objective was chosen to run, and the answer

Discovery's headline finding, recorded first, was that the incumbent wins on the literal reading:

Cosmic Bull already owns a live implementation that satisfies the literal objective: r/permission_registry. … Any consumer that accepts its trust model can use it today and write no new code at all.

That is written at the top of DISCOVERY.md, before the candidate matrix and before any argument for building — and it nearly ended the benchmark with no implementation. So the decline-to-build direction was reached: the agent found the incumbent, read all 526 lines of it, byte-verified it against the live chain, and recommended it.

What kept the question open is a quoted property, not a preference:

	MaxResources              = 1000
	MaxResourcesPerAdmin      = 20
	MaxPermissionsPerResource = 50
	MaxHoldersPerPermission   = 200

These caps are global, shared, and unraisable. A realm is immutable after addpkg, and those 526 lines contain no owner, no governance hook and no upgrade path (verified by full read, and by a repo-wide grep for mustBeAdmin|assertOwner returning that file only). The consequences are concrete, not theoretical: resource names are one flat first-come-first-served global namespace contended across every application; the registry closes permanently to new applications at 1000 resources; and an application needing a 201st holder of one permission has no recourse at all — it cannot raise the cap, cannot fork the realm, and sharding costs it another of its 20 slots.

So the finding on record is narrow and falsifiable:

The gap is not "nobody built this." It is that every implementation found puts permission state either in a shared realm with fixed global capacity, or in a library with no bounds and no authorization at all. Nothing found gives a consuming realm permission state that is simultaneously its own, bounded, and authorization-carrying.

Per the discovery rule the claim stays bounded — "no relevant existing implementation was found in the searched sources" — never ecosystem-wide uniqueness from a finite search.

The incumbent was not modified, not extended, and not redeployed. permission_registry's [addpkg] height is unchanged at 604615, alongside the other 16 siblings. Recommending a component and then leaving it alone is the half of the decision that is easy to skip.

The discovery method is part of the result

Five sources, 634 pearl-1 packages enumerated in full rather than swept by keyword. That choice paid for itself immediately: p/samcrew/basedao is a live, real role-and-member store and matches none of access|perm|rbac|role|auth|acl|grant. A keyword sweep would have missed it and the discovery record would have been wrong in a way no reviewer could see — which is exactly how the earlier feeledger / bazaar/fee/v1 miss happened.

Source 4 read the chain-matched corpus on disk (1,022 .gno files from the tree pearl-1 was built from), not master, so the corpus is the one the target chain actually runs.

The architecture — invert the ownership, and write no containers

permbook's single idea is that authorization state should be the consumer's own. Each consuming realm allocates its own *Book with its own Limits, so capacity contention between unrelated applications is structurally impossible rather than merely unlikely. The caps the incumbent could not raise become per-Book constructor arguments, clamped by ceilings the consumer cannot exceed.

The second decision was to write no container code at all. gno.land/p/nt/groups/v0 already provides address sets, a named-role registry, ordered iteration and readonly views — and, decisively, it already documents the three realm-boundary rules that govern handing a mutable handle across a boundary. permbook composes it and contributes only the policy layer: bounds, admin authority, name validation, pruning, and the query shape.

Two authorities, deliberately separate. Holding a permission out of a Book confers no power over the Book. That separation is what makes the capability safe to hand to an application that does not fully trust its own operators, and it is the property the live run was built to attack.

No coins, no effects, no chain APIs, no package-level state — the same posture as feeledger and duebook. The objective's instruction not to add economic mechanisms to a capability that does not need them required no restraint here: there was nothing to add.

What it cost in new code

One new /p/ primitive, one reference realm. 599 deployed code lines (271 + 328, excluding blanks and comments).

BasisNewDependencyNew share
whole imported packages5992,01023.0%
reached dependency surface5991,17033.9%

Both are reported because they differ by 11 points and either alone misleads. The dependency closure was measured from pearl-1, declaration by declaration, not from the local module cache: p/nt/groups/v0 (241), p/moul/addrset (61), p/nt/bptree/v0 (742), p/nt/avl/v0 (448), p/nt/markdown/sanitize/v0 (518). The reached column counts only the declarations permbook's call graph actually enters — all of bptree's insert/delete/rebalance core, but not its reverse iterators; avl's mutation path but not TraverseInRange; sanitize's InlineText/Blockquote closures but not the URL filters. 41,946 further bytes of test were written and are not deployed.

23.0% is the lowest new-code share of any factory benchmark (§7: 32–51%, §13: 37.5–52.7%). That is the intended direction: the run that found a strong incumbent and a strong primitive to compose should write the least code.

Verification

53 committed tests (36 primitive + 17 realm), all passing against the chain-matched toolchain and re-run from a fresh clone before deploy. Adversarial audit: 0 RED; findings Y1, Y2, O1, O2 and D1 all resolved.

O1 and O2 carry a disclosed caveat: the demo auditor could not dispatch the Task tool for its false-positive filter pass, so it ran the second pass in its own context and flagged its own anchoring bias. Both rest on a theory-of-unreachability judgment it marked as wanting independent confirmation. Recorded as a harness deviation rather than presented as a clean two-pass audit.

Live: 5 broadcast transactions across 2 identities, 6 adversarial refusals.

The central result. The sequence was run deny → grant → allow → revoke → deny, so both transitions are witnessed at real block boundaries rather than inferred from one of them:

Bump              ->  permission denied: bump      (admin, holds nothing)
Grant("bump", …)  ->  granted                       h612751
Bump              ->  bump 1 at height 612753       h612753
Revoke("bump", …) ->  revoked                       h612756
Bump              ->  permission denied: bump      (refused again)

The opening denial is the load-bearing one: the caller was the Book's own admin and was still refused. Administering a Book and holding a permission out of it are separate capacities — the design's whole security claim, shown in the first transaction rather than argued in a doc comment.

Two results that only a live run produces. After the revoke, PermissionCount() returned 0, not 1 — the empty bucket was pruned, so a Book's storage tracks live grants and not the history of grants. And Bumps() stayed at 1: the effect of an authorized call outlives the authority that permitted it. That second one is the non-obvious integrator semantics, and it is the kind of thing a test suite can assert but only a chain can make undeniable.

Escalation, attacked from a real second key. hunter, holding only bump, was refused on Grant and NominateAdmin with permbook: caller is not the admin, and on SetNotice with permission denied: notice. A holder is the party an application is most likely to under-trust and still expose; both "grant myself more" and "take the book" were closed, and permission A demonstrably did not reach effect B.

Guard ordering read off the error text, as in §13: Bump with send=1000ugnot aborted with this entrypoint does not accept coins even though that caller held no bump permission at all. So rejectStraySend provably runs before the permission check — a refusal never happens after taking custody of coins, and a test asserting only "it aborted" cannot tell the two orderings apart.

What it taught

Amended 2026-09-22: the distinction above is the durable lesson and it stands. Two specifics in it were wrong or incomplete. The addrset/v0 resolution is what the default (mainnet) remote serves, not what the pinned cache used for permbook's committed tests — this bullet named the wrong side. And the gap is no longer merely "recorded": it has been closed by measurement for all 17 deployed packages. See §10.

10. Dependency-closure audit — hardening the factory, not an application

Run 2026-09-22 on the user's instruction to harden the validation methodology before expanding the portfolio. Not an application benchmark: no /p/ or /r/ was created, nothing was deployed, and no live application was modified. The question was the one §9 exposed — how does Cosmic Bull prove that what it tested is what the chain will execute?

The standard, mechanism and residual limits are in ../docs/DEPENDENCY_CLOSURE.md; the portfolio result is amended into ../pearl/DEPLOYMENT.md.

The answer: byte-verifying the root was half a proof

Deploy-time byte verification proves the root matches the commit. It says nothing about the packages the root imports — which are compiled into every path the realm runs. Three toolchain mechanisms let the tested closure diverge silently: the fetch remote is derived from the import path's domain and is therefore mainnet regardless of deploy target; the module-cache marker is a 0-byte file with no hash or chain-id, so a poisoned cache is indistinguishable from a correct one and -remote-overrides on a warm cache is a no-op; and workspace-local packages shadow the chain with the fetcher explicitly nil.

The failure mode is a passing test

Reproduced live, and this is the whole finding in one line:

$ gno test pearl/p/permbook
ok      ./pearl/p/permbook 	0.57s

That ok came from a closure where 3 of groups/v0's 4 compiled files differ from pearl-1's and the addrset path does not exist on pearl-1 at all. A green test is not evidence about which chain's bytes it compiled.

What it taught

11. The application factory, fourth run — service_market (§15)

Objective: a decentralized service marketplace — providers register services, customers purchase with on-chain payment, work is resolved and payment released only after successful resolution, with protection against unauthorized settlement, double payment and stuck funds.

Strategy chosen: pure composition — one new realm, no new /p/. The novelty is a two-level state machine (service lifecycle × order lifecycle) over the already-audited coinio + feeledger + sanitize/v0. Deployed 2026-09-22, tx 9f9d9f42…, h628080; 11 live transactions across 2 keys, both orders driven terminal (one released, one refunded fee-free), 19 adversarial rejections, the realm drained to exactly 0ugnot, and conservation H = E + U + F + S exact at every checkpoint. Record: pearl/DEPLOYMENT.md §15; measured cost 5,886,642 ugnot (1.704× realm overhead).

12. The application factory, fifth run — subscriptions (§17)

Objective: provider-created subscription plans, per-period subscriber payment, tracked state and billing periods, cancellation, and structural prevention of unauthorized, duplicate and out-of-window payments, with obligations explicit on chain and deterministic renewal / expiration / cancellation rules.

Strategy chosen: pure composition again — one new realm over coinio + feeledger + sanitize/v0; the new logic is the windowed-renewal arithmetic (paidThrough-anchored, early bound = duplicate-payment guard, exclusive grace bound) and the plan/subscription state machine. Notable: the pre-deploy audit's two YELLOWs were both fixed in code (a most-recent index so cancellation keeps paid entitlement; boundary exclusivity so a renewal always buys at least one block) and re-verified before deploy. Deployed 2026-09-22, tx 60c23cd2…, h639086; 11 live transactions across 2 keys including two in-grace renewals and a permissionless expire by a non-subscriber, 15 adversarial rejections, the realm drained to exactly 0ugnot. Record: pearl/DEPLOYMENT.md §17; measured cost 5,419,174 ugnot (1.815× realm overhead).

What has not been benchmarked

Stated so the coverage is not over-read: