The question is not whether AI belongs in the device, but which part of it.
Anyone building or buying an edge device for a charging park eventually faces the question: should it include AI? In that form the question cannot be answered, because the word covers three very different tasks. Mapping known faults to known remedies is one task. Looking something up in a body of knowledge when the fault is not known is a second. And turning operating data into readable text, so that a human knows on Monday morning what happened over the weekend, is a third.
These three tasks have completely different requirements in terms of response time, traceability and computing power. Lump them together and you quickly arrive at the idea that a large language model has to run inside the cabinet – and therefore at hardware that belongs there neither economically nor thermally. Separate them and the architecture becomes simple, and sizing the controller almost follows by itself.
This article carries the separation through: which layer takes which task, why the language model in a charging park should never switch anything itself, and what is really needed in terms of memory and compute at the end.
Three tasks with three different sets of requirements.
The first part is the most boring and the most important. A large share of operating faults at charge points is known: a residual current in the DC monitoring, a failed firmware update, a dropped backend connection, an authorization that goes through without the vehicle granting charging permission. Such cases have fixed procedures. They belong in a deterministic table that is versioned and tested together with the firmware – no model, no search, no network. It is instant, it is auditable, and it works even when nothing else does.
The second part is the long tail: unknown or compound faults where context helps. What has happened at this site before, what quirk does this charge point model have, which firmware version had which side effect. That is a retrieval task. One detail matters: in the device the query is not written in everyday language, it is generated from structured data – error code, status notification, model designation, firmware level. In other words from precise technical terms. A compact keyword search over a local index is already very strong for exactly that; a vector database on site solves a problem you do not have in this shape.
The third part is the only task that genuinely requires a language model: turning facts into language. The daily report, the hint for the technician, the summary of a fault history. The benefit here is that a human understands faster – not that a machine decides faster.
- Known fault, known remedy: deterministic table, versioned and tested with the firmware.
- Unknown fault: retrieval from a compact local index, query generated from error code and status notification.
- Wording for humans: the only task that strictly requires a language model.
- Acting: belongs in none of these three layers, but in one of its own – see the next section.
The language model proposes, a deterministic layer decides.
If the device is meant not only to advise but to intervene, a convenience question turns into a safety question. A charging park controller protects a grid connection and distributes power across sub-circuits that often carry more than charge points. A wrong action means an interrupted charging session at best and a tripped main fuse at worst. A language model that switches directly is the wrong architecture here – regardless of how good its knowledge is.
The workable split is: the model proposes, a deterministic layer decides. Only what is on a positive list is permitted, every action with checked preconditions and every action reversible. Reducing a current limit, yes. A soft restart of a charge point with no active transaction, yes. Anything that raises load, touches protective functions or interrupts an ongoing billing record does not belong on that list – not even when the proposal sounds plausible.
This requires a complete journal: trigger, proposal, decision, outcome, with timestamps. That is not only for debugging. It is the basis for an operator trusting the automation at all, and it is the basis of any liability question. Automation that cannot be retraced gets switched off after the first unexpected intervention – and is therefore worthless.
Sizing follows from the architecture, not from the model size.
Once the separation is done, sizing is almost trivial. The rule set and the knowledge index cost practically nothing: a body of a few thousand entries is a file in the single-digit megabyte range, and searching it takes no measurable memory. Both run on the hardware that is in the cabinet anyway for the OCPP server and load management.
The only real cost driver is text generation, and there are exactly two answers for it. Either it happens centrally – then the site needs nothing for it beyond network access and an interface. Or it has to work offline, in which case a small model in the two to four billion parameter class is enough. Quantised, such a model occupies roughly two to three gigabytes of memory, runs without a graphics card on a modern CPU and answers within seconds. For a hint to a technician that is ample.
What does not belong in a cabinet are models in the thirty billion class. Around twenty gigabytes of memory and a graphics card turn that into a workstation rather than an appliance – with the corresponding waste heat, power draw and spare part question across a ten year plant lifetime. Such models make sense centrally, not on site.
- Rule set and knowledge index: a few megabytes, no measurable memory, runs on existing hardware.
- Central text generation: no additional site requirement beyond network access and an interface.
- Local offline text generation: small model, roughly 2 to 3 GB of memory, feasible without a graphics card.
- Rule of thumb: 8 GB of RAM carries rule set, index and a small model; 16 GB leaves room to grow.
- Not in the cabinet: thirty billion class models needing around 20 GB and a graphics card.
The daily report almost always belongs centrally.
The daily report is the simplest case and at the same time the best illustration of the split. It is not time critical, it does not have to work offline, and it benefits from a larger model because the wording gets better. That is precisely why it does not belong in the device.
The pattern is plain: the device sends the structured facts of the day over an interface to the central system – energy delivered, number and duration of sessions, faults encountered, load management interventions, and generation and import where relevant. The central system turns that into text and sends it out. The device itself knows no model and needs no model dependency.
The gain is more than saved hardware. There is one model to maintain instead of one per site. An improvement to the wording takes effect everywhere at once. And if the connection drops, the report arrives later – which is harmless, unlike load management, which is exactly why that has to stay local. This asymmetry is the real reason for the split: what cannot wait belongs on site.
The real gain is the feedback loop from the field.
Retrieval and wording are useful but replaceable. Something else is not: that every incident resolved at one site becomes knowledge for all sites. Whoever looks after thirty charging parks otherwise solves the same fault thirty times for the first time.
For that the loop has to be closed. The device reports what happened, what was proposed, what was carried out and how it turned out. Centrally that becomes a curated entry, and it goes to all devices with the next update. The separation of authoring and delivery matters here: curation happens centrally, in a tool people actually use. What gets delivered is a compiled, read-only index. That way no third party service, no per unit licence and no dependency that has to outlive its vendor remains in the field.
The entry point is therefore not a model but a list. Anyone who writes down their ten most frequent operating faults, together with the remedy that actually helped, has done the hard part. The rest is engineering – and can be added step by step without turning a charging park into a testing ground.