# Automated SRS Structural Standardization with LLMs
This repository provides scripts and data for the paper,
"*LLM-Based Structural Standardization of Software Requirements Specifications for ISO Conformance*."


## Overview
This repository includes:
- SRSs derived from the PURE dataset used in the study
- Scripts for LLM-based restructuring and automated evaluation
- Results from both automated and manual evaluations

- **`check/`**  
  Contains results of the manual evaluation.  
  For each target SRS, prompt, and LLM, this directory includes:
  - A selected restructured SRS from `out/`
  - Evaluation results from independent annotators  
  The evaluation checklist used in this process is also provided.

- **`data/`**  
  Contains SRSs derived from the PURE dataset used in the experiment.  
  This includes:
  - Original PDF files
  - Extracted text
  - Preprocessed versions used as model inputs

- **`out/`**  
  Contains SRSs structurally standardized by LLMs.  
  This includes:
  - Raw model outputs
  - Postprocessed outputs generated by our scripts
  - Results of automatic evaluation for each generated document

- **`srs2std/`**  
  Contains scripts for SRS restructuring and automated evaluation.  
  The `tests/` directory includes development tests for these scripts.


## How to Reproduce
We recommend using a virtual environment.
The Python package manager `uv` is recommended for dependency management.

Some components are omitted due to licensing restrictions, including:
- `iso_template.yaml`
- `srs2std/evaluate/decontextualize_strategies/molecular_prompts.py`

**To reproduce the experiments, these components must be supplemented appropriately.  
Refer to the corresponding files for guidance.**

### Setup
First-time setup:

```sh
$ echo OPENAI_API_KEY="YOUR_OPENAI_API_KEY" >> .env
$ echo GEMINI_API_KEY="YOUR_GEMINI_API_KEY" >> .env
$ echo GPTOSS_BASE_URL="ENTRY_POINT_FOR_GPTOSS" >> .env  # for decontextualization

# Not necessary if using Docker
$ uv sync
$ uv pip install -e .

# Download data required for NLTK sentence tokenization
# Not necessary if using Docker
$ uv run python -c "import nltk; nltk.download('punkt'); nltk.download('punkt_tab')"
```

### Execution Example
```sh
# Run all tests
$ uv run pytest

# Generate standardized SRS with "P_each" prompting
$ uv run srs2std generate data/01/preprocessed.md -p naive_each

# Evaluate standardized SRS againt its original version
$ uv run srs2std evaluate out/01/naive_each/gemini_2_5_flash data/01/preprocessed.md
```

### Through Docker
```sh
# Build and start container
$ docker compose run -it --rm --build app bash

# Run commands inside container
$ uv run pytest
$ uv run srs2std generate data/01/preprocessed.md -p naive_each
$ exit
```
