Skip to content

Commit f8d281b

Browse files
The9CatThe9Cat
authored
Merge pull request #188 from EXP-code/copilot/sub-pr-183-another-one
Add test coverage for HDF5 compression in EmpCylSL disk cache
2 parents 1072aa3 + 3295b99 commit f8d281b

File tree

4 files changed

+511
-1
lines changed

4 files changed

+511
-1
lines changed

‎.gitignore‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,11 @@
11

22
build/*
33

4+
# Python cache files
5+
__pycache__/
6+
*.py[cod]
7+
*$py.class
8+
49
Makefile
510
*.lo
611
*.a

‎tests/CMakeLists.txt‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,13 @@ if(ENABLE_PYEXP)
2626
${PYTHON_EXECUTABLE} cyl_basis.py
2727
WORKING_DIRECTORY "${CMAKE_CURRENT_LIST_DIR}/Disk")
2828

29+
# Test HDF5 compression functionality in cylindrical basis
30+
add_test(NAME pyexpCylCompressionTest
31+
COMMAND ${CMAKE_COMMAND} -E env
32+
PYTHONPATH=${CMAKE_BINARY_DIR}/pyEXP:$ENV{PYTHONPATH}
33+
${PYTHON_EXECUTABLE} cyl_compression_test.py
34+
WORKING_DIRECTORY "${CMAKE_CURRENT_LIST_DIR}/Disk")
35+
2936
# Remove cache files
3037
add_test(NAME removeSphCache
3138
COMMAND ${CMAKE_COMMAND} -E remove .slgrid_sph_cache
@@ -45,7 +52,7 @@ if(ENABLE_PYEXP)
4552

4653
# Set labels for pyEXP tests
4754
set_tests_properties(pyexpLoadTest pyexpSphBasisTest PROPERTIES LABELS "quick")
48-
set_tests_properties(pyexpCylBasisTest PROPERTIES LABELS "long")
55+
set_tests_properties(pyexpCylBasisTest pyexpCylCompressionTest PROPERTIES LABELS "long")
4956

5057
endif()
5158

‎tests/Disk/README.md‎

Lines changed: 119 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,119 @@
1+
# HDF5 Compression Tests for EmpCylSL
2+
3+
## Overview
4+
5+
This directory contains tests for the HDF5 compression functionality implemented in the EmpCylSL cylindrical disk basis cache system.
6+
7+
## Test File: `cyl_compression_test.py`
8+
9+
This comprehensive test suite validates the HDF5 compression implementation by testing:
10+
11+
### Test 1: Compression Disabled
12+
- Verifies that compression can be disabled by setting `compress: 0` in the YAML configuration
13+
- Creates a basis with no compression and verifies the cache file is created and readable
14+
15+
### Test 2: Compression Enabled
16+
- Tests multiple compression levels (1, 5, 9)
17+
- Verifies that each compression level works correctly
18+
- Ensures cache files are created and readable for all compression levels
19+
20+
### Test 3: Compressed File Readback
21+
- Creates a compressed cache file with compression level 5
22+
- Verifies the file can be read back correctly by a second basis instance
23+
- Ensures data integrity is maintained through compression/decompression
24+
25+
### Test 4: Compression Level Validation
26+
- Tests valid compression levels (0, 5, 9)
27+
- Tests out-of-range compression levels (10, 15, 100)
28+
- Verifies that out-of-range values are clamped to 9 as implemented in `setH5Params`
29+
30+
### Test 5: Shuffle Parameter
31+
- Verifies the shuffle filter works in conjunction with compression
32+
- Note: The shuffle parameter is enabled by default in `setH5Params` when compression is enabled
33+
- Tests indirectly by verifying successful cache creation with compression
34+
35+
### Test 6: Compression Comparison
36+
- Compares file sizes between uncompressed (level 0) and compressed (level 9) cache files
37+
- Provides visual feedback on compression effectiveness
38+
- Note: Small test datasets may not compress significantly
39+
40+
## Running the Tests
41+
42+
### Via CMake/CTest
43+
44+
After building the project:
45+
46+
```bash
47+
cd build
48+
ctest -R pyexpCylCompressionTest --output-on-failure
49+
```
50+
51+
Or as part of the full test suite:
52+
53+
```bash
54+
ctest -L long --output-on-failure
55+
```
56+
57+
### Direct Execution
58+
59+
You can also run the test directly if pyEXP is built and in your PYTHONPATH:
60+
61+
```bash
62+
cd tests/Disk
63+
python3 cyl_compression_test.py
64+
```
65+
66+
## Expected Output
67+
68+
When all tests pass, you should see output similar to:
69+
70+
```
71+
======================================================================
72+
EmpCylSL HDF5 Compression Test Suite
73+
======================================================================
74+
75+
Test 1: Testing compression disabled (compress=0)...
76+
✓ Compression disabled test passed
77+
78+
Test 2: Testing compression enabled with different levels...
79+
Testing compression level 1...
80+
✓ Compression level 1 test passed
81+
Testing compression level 5...
82+
✓ Compression level 5 test passed
83+
Testing compression level 9...
84+
✓ Compression level 9 test passed
85+
✓ All compression level tests passed
86+
87+
[... more tests ...]
88+
89+
======================================================================
90+
Test Summary
91+
======================================================================
92+
Passed: 6/6
93+
✓ All tests passed!
94+
```
95+
96+
## Implementation Details
97+
98+
The compression functionality is implemented in:
99+
- `src/Cylinder.cc`: Reads the `compress` parameter from YAML config and calls `setH5Params`
100+
- `include/EmpCylSL.H`: Defines `setH5Params` method with validation
101+
- `exputil/EmpCylSL.cc`: Applies compression settings when writing HDF5 cache files
102+
103+
The `setH5Params` method signature:
104+
```cpp
105+
void setH5Params(unsigned compress, bool shuffle=true, bool szip=false)
106+
```
107+
108+
Where:
109+
- `compress`: Gzip compression level 0-9 (0=off, 9=max)
110+
- `shuffle`: Enable byte shuffle filter (default: true)
111+
- `szip`: Enable szip compression (default: false)
112+
113+
## Test Coverage
114+
115+
These tests address the feedback from PR #183:
116+
- ✅ Compression can be enabled/disabled via the compress parameter
117+
- ✅ Compressed cache files can be read back correctly
118+
- ✅ The shuffle parameter works as expected
119+
- ✅ Validation of the compression level (0-9 range)

0 commit comments

Comments
 (0)