Computer Science > Software Engineering
Title:A Chain-of-Thought Prompt Engineering Framework for Automated Technical Specification Generation in Enterprise Integration: A Real-World Case Study
Abstract:Technical Specification Documents (TSDs) translate functional requirements into implementable designs and are mandatory artifacts in governed Enterprise Application Integration (EAI) programs, yet their production remains a manual, expertise-intensive bottleneck. Large Language Models (LLMs) have been studied extensively for code generation, but long-form, template-governed engineering documents remain comparatively unexplored. This paper reports an industrial case study, conducted during a migration of more than 250 enterprise integrations to Microsoft Azure Integration Services at a large Australian wholesale distributor, that introduces a chain-of-thought prompt-engineering framework for automated TSD generation using GitHub Copilot. The framework combines a two-phase generation architecture that grounds the model in the functional specification, organizational standards, and the approved template; a six-stage prompting pipeline with human validation checkpoints; and Prompt-as-a-Code, under which prompts are version-controlled and reviewed as engineering artifacts. Applied to a stratified sample of fifteen production interfaces, the framework reduced TSD authoring effort from 793 to 300 person-hours, a 62.2% pooled reduction and a 64.0% mean per-document reduction (Wilcoxon signed-rank, n = 15, p < 0.001), at 74.3% reviewer-assessed accuracy and 3.7/5 developer satisfaction. Benefits degrade monotonically with interface complexity: between the simplest and most complex bands, effort reduction falls from 70.6% to 58.9% and accuracy from 91% to 62.5%, and the residual human effort concentrates in diagrams, sequence flows, and cross-system integration content. We additionally report comparative observations of GPT-4.1/5 and Claude Sonnet 4, as well as Copilot's Ask and Agent interaction modes, offering reusable, practice-grounded guidance for enterprise settings.
| Comments: | 15 pages, 10 figures, |
|---|---|
| Subjects: | Software Engineering (cs.SE) |
| ACM classes: | I.2.7; D.2.2; D.2.9 |
| Cite as: | arXiv:2611.21617 [cs.SE] |
| (or arXiv:2611.21617v1 [cs.SE] for this version) | |
| https://doi.org/10.48550/arXiv.2611.21617 arXiv-issued DOI via DataCite |
Submission history
Access Paper:
Additional Features
- Audio Summary
Current browse context:
References & Citations
- Google Scholar
- Semantic Scholar
BibTeX formatted citation
Bookmark
Bibliographic and Citation Tools
Code, Data and Media Associated with this Article
Demos
Recommenders and Search Tools
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.