How to Install AI On-Premise in Your Company?
This guide explains step-by-step how to deploy generative artificial intelligence in your company without sending your data to the cloud. It covers the 3 possible architectures, the 5 concrete deployment steps, costs, GDPR and AI Act compliance, and common mistakes.
Why Install AI On-Premise Instead of Using ChatGPT?
Four concrete reasons drive French companies to deploy AI on their own premises rather than use an online service.
Data confidentiality
With ChatGPT, Claude, or Gemini, every question sent by an employee transits through foreign servers (primarily United States). For a law firm, a hospital, or a company under professional secrecy, this is legally risky and often prohibited by professional codes of conduct.
GDPR and AI Act compliance
Since February 2025, the European AI regulation (AI Act) requires complete traceability of any AI system used in a company. An AI installed on-premise is natively auditable by the DPO and CNIL. A cloud AI requires complex subcontracting agreements and specific impact assessments.
Cost at scale
ChatGPT Enterprise costs $65 per user per month. For 100 employees: $78,000 per year, or $234,000 over 3 years. An AI installed on-premise costs once ($27,000 to $98,000 depending on size), then annual maintenance ($5,400 to $16,300). Users are unlimited at no extra cost.
Offline availability
An AI installed on-premise works even when the internet is down. For factories, remote sites, classified environments (defense, healthcare, sensitive public sector), this is non-negotiable.
The 3 Possible Architectures for Deploying AI in a Company
Before choosing how to install AI internally, you need to understand the 3 main options. Each has a different cost, level of confidentiality, and installation effort.
| Criterion | Cloud AI (ChatGPT, Claude…) | On-premise AI (on-site) | Hybrid architecture |
|---|---|---|---|
| Where does your data go? | Vendor servers (often US) | Stays within your internal network | Sensitive data on-prem, rest in cloud |
| Cost for 100 users over 3 years | $165,000 to $240,000 (per-seat subscription) | $65,000 to $100,000 (hardware + service) | $88,000 to $165,000 (combined) |
| GDPR / AI Act compliance | Complex: data processing agreements, impact assessments | Native: everything under your control | Varies depending on data split |
| Installation time | Immediate (account creation) | 10 to 15 days (delivery + configuration) | 1 to 3 months (mixed architecture design) |
| Works offline? | No | Yes | Partially |
| Best suited for | Startups, freelancers, low-sensitivity data | Law firms, healthcare, defense, mid-sized companies under GDPR constraints | Large enterprises with mixed use cases |
The 5 steps to install AI on-premise in your company
Here are the 5 concrete steps to deploy generative AI on-site in your company. This process applies whether you use a turnkey vendor (like LMbox) or build the solution yourself.
-
1
Step 1: Define the scope of use
List the 3 to 5 priority use cases (writing, document analysis, internal search, meeting summaries, etc.) and identify the document sources to connect (SharePoint, shared folders, email, etc.). Also define target users and the required confidentiality level.
Durée : 2 to 5 days -
2
Step 2: Select hardware and AI model
Size the hardware according to the number of concurrent users. For 5 to 15 people: a compact appliance is sufficient (processor card + 128 GB RAM + SSD). For 30 to 100 people: a rack-mounted server with a graphics card. The AI model (Mistral, Llama, Gemma, etc.) runs locally on this hardware.
Durée : 1 to 2 days -
3
Step 3: Install hardware and software
The hardware is installed in your technical room or server room. The AI software (models + interface + connectors) is installed. If you use a turnkey vendor, this step is completed by their team in 1 to 3 days on-site. If you do it yourself, allow 2 to 4 weeks.
Durée : 1 to 3 days (turnkey) or 2 to 4 weeks (DIY) -
4
Step 4: Connect your document sources
The AI must access your internal documents to provide useful answers. Connect your document repository (SharePoint, Google Drive, NetDocuments, etc.), your email, and possibly your business tools. Permissions from your directory (Active Directory, Azure AD, Google Workspace) are inherited: an intern cannot see partners' folders.
Durée : 1 to 5 days depending on the number of sources -
5
Step 5: Train teams and monitor usage
Employees access the AI from their browser, using their usual credentials. Training is brief (30 minutes to 2 hours), as the interface resembles ChatGPT. An admin dashboard tracks usage, performance, and provides a complete audit log (useful for compliance).
Durée : 1 to 3 days
The 4 questions to ask before choosing
Is your data subject to professional secrecy, strict GDPR, or sector-specific regulations (healthcare, defense, finance)?
If yes, on-premise AI is almost always mandatory. ChatGPT and Claude are not compatible with attorney-client privilege, medical confidentiality, or HDS (Health Data Hosting) regulations.
How many employees will use the AI?
Below 10 users, cloud subscriptions remain financially competitive. From 20 to 30 users, on-premise AI becomes cheaper over 3 years. At 100 users and above, the gap widens significantly in favor of on-premise.
Do you have an IT team to manage the installation?
If yes, you can choose a self-assembled solution (open-source: Ollama, vLLM, LiteLLM). If not, choose a vendor that delivers a turnkey solution (hardware + software + support).
Do you need the AI to work without internet?
If you work on isolated sites (factories, ships, electrical substations) or in classified environments, choose a solution capable of running in fully isolated mode (air-gap). Not all on-premise AI solutions support this.
How much does on-premise AI cost in 2026?
Costs depend on the number of users and the desired service level. Here are typical ranges observed in the French market in 2026.
| Company size | Hardware (one-time purchase) | Annual service | 3-year total | Concurrent users |
|---|---|---|---|---|
| 5 to 15 people (SMB, firm) | €20,000 to €30,000 | €5,000 to €8,000 | €35,000 to €55,000 | 10 |
| 30 to 80 people (mid-market, medium organization) | €25,000 to €40,000 | €6,000 to €10,000 | €45,000 to €70,000 | 25 |
| 100 to 300 people (mid-market, multi-BU) | €50,000 to €90,000 | €12,000 to €20,000 | €85,000 to €150,000 | 100 |
For comparison, ChatGPT Enterprise costs €60 per user per month (OpenAI public pricing, May 2026). For 100 people over 3 years, that represents €216,000 versus €85,000 to €150,000 for an on-premise AI.
GDPR and AI Act compliance: what changes with on-premise AI
The European AI Act came into force in February 2025. It requires full traceability of AI systems used in enterprises. On-premise AI meets these obligations natively, unlike cloud AI.
GDPR: no data transfer outside the EU
With on-premise AI, your data never leaves your internal network. No need for impact assessment for transfers outside the EU, no need for standard contractual clauses. GDPR is respected by design.
AI Act: native traceability and auditability
Every action (agent call, document read, configuration change) is logged locally with cryptographic timestamping. The DPO and CNIL can audit the system at any time without special preparation. Penalties avoided: up to €35M for non-compliance.
HDS (healthcare): native compatibility
For healthcare facilities, on-premise AI on HDS-certified hardware directly meets the obligations of Article L1111-8 of the Public Health Code. No need for a third-party certified host.
Professional secrecy: technically preserved
For lawyers (Article 5 of the CNB internal regulations), notaries, physicians, and chartered accountants, on-premise AI technically preserves professional secrecy. Cloud services do not allow this without exemption.
5 common mistakes to avoid when deploying AI in-house
-
1
Underestimating the work on document sources
An AI is only useful if it can access your internal documents. If your employees store their files individually without a naming convention, you must first centralize. Plan 1 to 3 months of cleanup before deployment.
-
2
Over-sizing hardware as a precaution
Many companies purchase over-sized hardware "just in case." Result: 30% to 50% cost overrun for capacity that is never used. Size according to the number of actual concurrent users, not total headcount.
-
3
Not appointing an internal owner
Without a designated person on the company side (IT manager, CIO, or even a tech-savvy partner), the project stalls after installation. Appoint this owner from the start, even before ordering.
-
4
Forgetting usage governance
Without an internal usage policy (who can access what, on which topics, with what obligations), employees create their own practices. Establish the policy before deployment, even if it evolves later.
-
5
Skipping user training
The interface resembles ChatGPT but best practices differ (how to formulate a request properly, how to verify a response, when to switch to a human). 30 minutes to 2 hours of training increases adoption by 40% to 80%.
Frequently asked questions about deploying AI in-house
How much does in-house AI cost for an SME? +
For an SME of 5 to 15 people, typical cost is $20,000 to $30,000 for hardware (one-time purchase, 3-year warranty), plus $5,000 to $8,000 per year for service (updates, support, new AI models). Total of $35,000 to $55,000 over 3 years, unlimited users. Compare to a ChatGPT Enterprise subscription ($60 per user per month, or $21,600 to $32,400 over 3 years for 10 to 15 people), with the advantage that data never leaves.
Is in-house AI compliant with GDPR and the AI Act? +
Yes, by design. Since no data leaves the company network, GDPR does not require an impact assessment for transfers outside the EU. The AI Act, which came into force in February 2025, requires complete traceability of usage: in-house AI natively logs every action (user, timestamp, document accessed), exportable for audit by the DPO or data protection authority. Cloud AI requires complex subcontracting agreements and specific impact assessments.
Is in-house AI as capable as ChatGPT? +
For everyday tasks (writing, analyzing a contract, summarizing a document, searching archives), the level is comparable. Recent open AI models (Mistral, Llama, Gemma, Qwen) rival the best cloud services. And because in-house AI knows your internal documents, its responses are often more accurate than a generic service. For very complex tasks requiring very broad general knowledge, the largest cloud models still have a slight edge, but the gap narrows with each new version.
Does in-house AI work without internet? +
Yes, completely. The hardware and AI model run locally, without calling an external service. This is one of the major arguments for isolated sites (factories, ships, substations), classified environments (defense, healthcare), and organizations that cannot afford an internet outage. Fully isolated mode ("air-gap") is even mandatory for certain sensitive sectors (critical infrastructure operators).
Do you need an IT team to install and maintain in-house AI? +
No, not necessarily. If you choose a turnkey solution (hardware + software + support), installation is done by the vendor in 1 to 3 days on-site. Routine maintenance (updates, backups, new models) is then managed remotely by the vendor. A part-time IT manager is sufficient to oversee everything from a dashboard. If you choose a solution to assemble yourself, plan 1 to 2 full-time IT staff for 2 to 4 weeks for installation, then 0.5 FTE for maintenance.
How long until teams are operational? +
With a turnkey solution, plan 10 to 15 days between order and first real use by your employees: 5 to 7 days for hardware delivery, 1 to 3 days for on-site installation and configuration, 2 to 5 days to connect your document sources (SharePoint, Drive, email). User training takes 30 minutes to 2 hours per person. With a solution to assemble yourself, plan 2 to 4 months between decision and production deployment.
Available solutions for deploying AI in-house in 2026
Several approaches exist for deploying generative AI in the enterprise. Here are the main options, ranked by level of support.
Turnkey solutions (hardware + software + support)
The vendor delivers a pre-loaded appliance or server, installs it on-site, configures connections to your tools, trains your teams. Example: LMbox offers an AI appliance installed in your premises in 10 to 15 days, with 3 formats (Compact, Rack, Rack Pro) for 5 to 300 users, starting at $22,000 + $6,000 per year.
Software solutions to install on your hardware
You purchase your hardware separately (often a server with graphics card) and install commercial software (NVIDIA NIM, Anyscale, etc.) or open-source (Ollama, vLLM, LiteLLM, Open WebUI). More economical on hardware, but requires a skilled IT team and 2 to 4 weeks of installation.
Fully open approach (DIY)
You assemble all components yourself: hardware, AI model (Mistral, Llama, Gemma downloaded from Hugging Face), inference server (vLLM, llama.cpp), user interface (Open WebUI, LibreChat), document connectors. Lowest upfront cost, highest internal time investment. Relevant for IT teams that want total control.
LMbox: an AI installed on your premises, operational in 15 days
LMbox is a generative AI appliance installed on your premises. Your teams use it like ChatGPT—but no data leaves your network. Designed and supported in France, GDPR-compliant by design, HDS-compatible, operational in 10 to 15 days.
- 3 formats selon votre taille (5 à 300 utilisateurs)
- Installation et formation comprises (3 jours sur site)
- Connexion aux outils existants (SharePoint, Drive, messagerie)
- Journal d'audit complet et exportable pour le DPO et la CNIL
- Mode air-gap disponible pour les environnements sensibles
Go further
Three concrete resources to learn more.
The LMbox product page
Details on the appliance, the 3 formats, pricing, installation requirements.
View the LMbox pageThe cost calculator
Estimate the 3-year cost of an on-premise AI for your company size.
Calculate my costUse cases by function
15 real-world AI use cases in the enterprise, organized by sector (legal, healthcare, manufacturing, HR, compliance, etc.).
View use cases