Cloudflare/Security-Audit-Skill

(github.com)

205 points | by donk8r 1 day ago

10 comments

  • gbrindisi 1 day ago
    Shameless plug: in case someone finds this requiring too many tokens, we shared the recipe on how we built our own in house audit skill so that it can easily be replicated and tuned to different environments https://www.synthesia.io/post/automating-code-security-revie...
  • prodigycorp 1 day ago
    Hi Cloudflare people, if you are reading this. Please clean up your Cloudflare. Skills. There are way too many skills for the platform. You should consolidate all of your skills into a single skill and route everything thru that skill. The way it is right now pollutes our context window.

    https://github.com/cloudflare/skills/tree/main/skills

    • simlevesque 1 day ago
      14 skills with short descriptions doesn't seem like an issue to me.
      • riffic 22 hours ago
        a skill issue.
      • prodigycorp 1 day ago
        The descriptions are shorter now, but they used to read like ads. The skill split still feels arbitrary. You turn on what you think you need, then hit some Cloudflare task that needs another skill and have to go fetch it.

        I don’t get why these need to be separate. One skill with progressive disclosure already handles this. That’s kind of the point.

    • mooktakim 1 day ago
      I really don't understand why they didn't create an "app store" style skills library where it could be approved and scanned for security issues
    • m00dy 1 day ago
      I'm sure they read here.
  • drchaim 1 day ago
    I threw 1M tokens for nothing in a medium codebase.
    • SkyPuncher 1 day ago
      These work best on a targeted section of the code, like a PR.
    • TZubiri 1 day ago
      how much is medium codebase, like 50kloc including docs?
      • drchaim 1 day ago
        in this case medium is relative to the projects I've worked. Bad expression anyway.
      • throwup238 1 day ago
        500kloc plus at least ten million lines of gastown logs.

        For a todo cli. That doesn’t work.

        • this_user 1 day ago
          Welcome to agentic coding in 2026.
    • TrustScoreAgent 1 day ago
      [dead]
  • qsbuilder 22 hours ago
    Dumping 14 full schemas into the prompt is just lazy design. You burn tokens, spike latency for no reason
  • wslh 1 day ago
    Tip for security professionals using LLMs: audit skills that explicitly frame the task as security research sometimes trigger refusals from the top OpenAI and Anthropic models because they guard against misuse. What works for me: separate skills for bug classes (and bugs in general) without the security framing, plus another skill that combines their findings to spot security bugs.
    • viraptor 1 day ago
      If you're a security professional, go through their validation. You won't get the security refusals anymore. Well... you'll still get the occasional downgrade from Fable, but not the "oh no, I can't do exploits for you" breaks.
      • xur17 1 day ago
        Except their validation doesn't seem to work. I've gone through both (both personally and for my company), and.. no response for weeks.
        • viraptor 4 hours ago
          Weird. I have minimal publicly visible record of security work and got the approval almost immediately.
  • hyperionultra 1 day ago
    Uf, how much tokens?
    • jesse_dot_id 1 day ago
      At least 150k on my relatively small FastAPI project, but hit my session limit. Continuing in a few hours.
      • chrisweekly 1 day ago
        Oof. YAGNI. 150k tokens is where you start hitting the "dumb zone" (model attention issues and inconsistent adherence to instructions).
  • acedTrex 23 hours ago
    Incredible, a post and repo dedicated to a markdown file, the downfall of this field has been swift.
    • decidu0us9034 21 hours ago
      but they're very huge markdown files. look how much junk they're polluting the conext window with.
    • vntok 23 hours ago
      Did you open the repo? There's a subdirectory with two dozens of files, around 300Kb of text.

      Storing/visualizing small text changes over time as revisions is exactly what Git is excellent at, how else would you keep track of updates to the prompts?

      • acedTrex 23 hours ago
        > how else would you keep track of updates to the prompts

        I dont? because prompts are not a thing that are ever needed to be tracked lol.

  • 9el 1 day ago
    Any clues why "an OS-enforced sandbox" is in requirements?
    • nicce 1 day ago
      Probably to save their skin if agent starts to do some unexpected things and bringing havoc. But I doubt that OpenAI models with normal subscription, for example, wont even work with this skill.
    • donk8r 22 hours ago
      Runs target builds, tests, fuzzers. No sandbox: workflow won't execute them. Lead stays needs_validation.
    • awss1i 5 hours ago
      [flagged]
  • tonymet 1 day ago
    What’s the difference between a skill and a prompt? Separate files? Aren’t tokens, tokens?
    • fassssst 23 hours ago
      Skills can have scripts packaged with them
  • aitoolcrux 1 day ago
    [flagged]