GLM Built Its Own Inference Infrastructure

(z.ai)

47 points | by whiteros_e 2 hours ago

6 comments

  • dada216 32 minutes ago
    We built a complete production-grade inference service from scratch on a cluster of more than 100,000 Chinese-made AI accelerators. All production inference for GLM-5.3-Flash runs on this system.
  • tefkah 21 minutes ago
    > Today, GLM-5.3 has become an indispensable daily coding partner for everyone on the team, and it is moving steadily toward replacing us. If this trend continues, given enough compute and enough time, its endpoint is a system that can design and train its own successor entirely autonomously. This is known as Recursive Self-Improvement, or RSI.

    Statements dreamed up by the utterly deranged.

    • binsquare 6 minutes ago
      Given the rate of improvement, why is this deranged?
  • embedding-shape 29 minutes ago
    I was gonna ask how people found their coding plans, and realized, have they massively ramped up the prices? Seems the middle plan is ~$80/month now, didn't that used to be like $20/month? Cheapest plan is ~$20/month currently.

    They must have hit really hard scaling limits if the prices were hiked so much so quickly.

    • Daviey 16 minutes ago
      I paid $360 annual for Max plan and currently averaging about 1BN tokens a day with their frontier GLM-5.3 model. This was clearly unsustainable for them and they've dropped this package.
      • disiplus 6 minutes ago
        I also have a legacy pro plan and the only limitation is if you are trying to work in the morning from Europe because you are in the 3x usage overlapping China time but after 12 or so you basically can run it at least for me at least 3 parallel sessions all the time.
    • asp_hornet 15 minutes ago
      The way I look at it, their coding plan doesn’t retain data or use it for training making it one of the cheaper plans for me.

      https://docs.z.ai/legal-agreement/privacy-policy

      • andy_ppp 13 minutes ago
        You believe any of these companies care about the law? They care about winning and building the self improving AI as quickly as possible.
        • asp_hornet 10 minutes ago
          I too am sceptical but I’ll take my chances. At least it’s helping the open weights.
    • broodbucket 26 minutes ago
      Yeah it went from a great deal to unviable compared to other providers imo. They really need to find a healthy middle ground
      • lompad 12 minutes ago
        It just gives a taste of what we are all going to have to pay soon, once the model providers actually have to make money. And the era of "let's charge a dollar for every 10 dollars running the infra actually costs" is rapidly coming to an end.

        And you can bet GLM is still ridiculously subsidized, just not as ridiculously as Anthropic and OpenAI.

        • chobbledotcom 4 minutes ago
          This isn't true, you can pay for GLM 5.3 from a provider like Neuralwatt or Friendli who have no incentive to subsidize or loss-lead their inference APIs
      • pyrophane 12 minutes ago
        What provider are you using currently?
    • bbor 17 minutes ago
      It's hard to know, since no one advertises the actual token limits (partially cause they're prolly complex / adaptive). So it seems much more likely that they just offer different pricing tiers than you're used to. Like, the $80 plan is still ~$80 of subscription quota, regardless of what else is offered.

      For [API usage](https://openrouter.ai/z-ai/glm-5.3-flash#providers) they charge a bit more than the very cheapest providers of GLM-5.3-Flash, but not so much that a big price difference would make sense.

  • OhNoNotAgain_99 11 minutes ago
    [dead]
  • bbor 21 minutes ago
    Well, other than the infrastructure they got from illegally routing millions of paying customers' requests through Anthropic's Opus 4.8 in a distillation attack...
    • jensb1 18 minutes ago
      What is "illegal" about it?
      • bingud 11 minutes ago
        breaking Anthropic TOS and misleading users
      • bbor 5 minutes ago
        Are you joking...? Sorry if so! Just in case: It's illegal in both the PRC and the USA.

        In the PRC, they[1] leaked tons of national secrets on the PRC's latest AI campaigns, the inner workings of their "opinion monitoring" (read: performative panopticon) and "stability" (read: violent oppression) departments, Chengdu's whole CCTV network, direct-energy weapons plans, espionage activities in Syria to hunt down Uyghur refugees, and god knows what else that Anthropic didn't divulge to us common folk.

        In the US, it's very clearly an attempt to rip off a competitor. I'm not sure how else you could possibly see it. Even if you're a distillation fan in general (which A. why and B. plz don't), they did this through a network of Japanese and Signaporean shell accounts, presumably at least some of which were abusing Anthropic's subscription service in a ToS double-whammy, as it would be exorbitantly expensive otherwise. They also had to hack around Anthropic's API to get CoT traces, which seems impossible to explain away as anything innocent.

        I've been beating the "China isn't necessarily an enemy, it's gonna take us all to handle AI" drum for literally years, but this attack was just... gross. Gross in scale and gross in arrogance. Not a good sign for the dawning alignment crisis, to say the least :(

        TL;DR: Use these services if you want, but know that you're supporting aggressive escalations and companies that very clearly don't give a flying fuck about violating the law, much less your ToS. So... buyer beware, I guess.

        [1]: For clarity, Z.ai was not alone in this, nor were they most egregious attack -- Moonshot.ai (kimi) took that coveted prize. DeepSeek was involved, too.

        • Bluestein 0 minutes ago
          Nulla poena sine lege?
    • Laurel1234 10 minutes ago
      [dead]
  • rob74 4 minutes ago
    This article left me with one immediate question: "WTF is GLM?".

    Honestly, I have no idea what z.ai is either (I'm aware of an AI-enabled editor called Zed, but that's under zed.dev), so it's a bit presumptuous from them to assume that everyone is familiar with their product...