Add support for generating HiPS and reprojecting RGB images with AVM metadata by astrofrog · Pull Request #509 · astropy/reproject · GitHub
Skip to content

Add support for generating HiPS and reprojecting RGB images with AVM metadata#509

Merged
astrofrog merged 36 commits into
astropy:mainfrom
astrofrog:hips
Jul 4, 2025
Merged

Add support for generating HiPS and reprojecting RGB images with AVM metadata#509
astrofrog merged 36 commits into
astropy:mainfrom
astrofrog:hips

Conversation

@astrofrog

@astrofrog astrofrog commented Jun 17, 2025

Copy link
Copy Markdown
Member

This adds initial support for HiPS. Main TODOs:

  • reproject_to_hips for 2D celestial images
  • Function to combine/mosaic multiple different HiPS surveys
  • Tests
  • Documentation

Beyond this PR:

@astrofrog

Copy link
Copy Markdown
Member Author

@codecov

codecov Bot commented Jun 17, 2025

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 79.41176% with 63 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.86%. Comparing base (27aee71) to head (ce6538c).
⚠️ Report is 123 commits behind head on main.

Files with missing lines Patch % Lines
reproject/hips/high_level.py 74.31% 47 Missing ⚠️
reproject/common.py 66.66% 7 Missing ⚠️
reproject/utils.py 88.23% 6 Missing ⚠️
reproject/mosaicking/coadd.py 66.66% 2 Missing ⚠️
reproject/hips/utils.py 97.50% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #509      +/-   ##
==========================================
- Coverage   91.45%   88.86%   -2.59%     
==========================================
  Files          25       28       +3     
  Lines        1088     1374     +286     
==========================================
+ Hits          995     1221     +226     
- Misses         93      153      +60     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread reproject/hips/core.py Outdated
reproject_function,
output_directory,
tile_size,
progress_bar=None,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can't be None, gives TypeError: 'NoneType' object is not callable

@keflavich

Copy link
Copy Markdown
Contributor

The current approach doesn't produce the index.html file:

reproject_to_hips('/Users/adam/work/gc/ACES/ACES_tour_Aladin/12m_continuum_commonbeam_circular_reimaged_mosaic_MUSTANGfeathered_zerod.fits', coord_system_out='galactic', level=4, reproject_function=reproject_interp, output_directory='REPROJECT_TO_HIPS_MUSTANG_EXPERIMENT', progress_bar=tqdm)
$ ls REPROJECT_TO_HIPS_MUSTANG_EXPERIMENT
Norder0/    Norder1/    Norder2/    Norder3/    Norder4/    properties

@astrofrog

Copy link
Copy Markdown
Member Author

@keflavich - I've pushed some new changes. This now supports reprojecting from PNG/JPG with AVM. Here is an example:

import os
import shutil

from tqdm.notebook import tqdm
from reproject.hips import reproject_to_hips
from reproject import reproject_interp

filename = "MUSTANG_12m_feather_noaxes.png"
output_directory = filename.replace('.png', '_hips')

if os.path.exists(output_directory):
    shutil.rmtree(output_directory)

reproject_to_hips(filename,
              coord_system_out='galactic',
              level=6,
              reproject_function=reproject_interp,
              output_directory=output_directory,
              progress_bar=tqdm)

This now also outputs an index.html file, so if I serve the output directory and open it in the browser, I get:

image

@astrofrog

Copy link
Copy Markdown
Member Author

Ok I've now added a new coadd_hips function which can be add to combine multiple HiPS maps in the same coordinate frame (but potentially different order) into a single HiPS dataset:

from reproject.hips import coadd_hips
coadd_hips(['MUSTANG_12m_feather_noaxes_hips', 'MUSTANG_12m_feather_noaxes_offset_hips'], 'test_coadd')

Currently if there is overlap, the last image is shown on top. We could have some more fancy blending in future.

Comment thread reproject/hips/core.py Outdated
Comment on lines +75 to +76
level : int
The number of levels of FITS tiles.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would be helpful to auto-detemine the level from the input WCS by default

@keflavich

Copy link
Copy Markdown
Contributor

I'm not sure the coordinate_system_output is working; your example screenshot and my local reproduction both put it in celestial coordinates, not galactic

@keflavich

Copy link
Copy Markdown
Contributor

I have an example RGBA file. Can we handle those? They have a 4th layer. Does Aladin support transparency?

@keflavich

Copy link
Copy Markdown
Contributor

coadd didn't work:

from reproject.hips import coadd_hips
coadd_hips(['rgb_final_uncropped_hips', 'MUSTANG_12m_feather_noaxes_hips'], 'AshFigureWithACES')

Traceback (most recent call last):
  File ~/repos/avm_images/python_reproject_to_hips.py:58
    coadd_hips(['rgb_final_uncropped_hips', 'MUSTANG_12m_feather_noaxes_hips'], 'AshFigureWithACES')
  File ~/repos/python-reprojection/reproject/hips/core.py:360 in coadd_hips
    raise NotImplementedError()
NotImplementedError

most likely because one of these is jpeg and the other png

@keflavich

Copy link
Copy Markdown
Contributor

with pngs:

Traceback (most recent call last):
  File ~/repos/avm_images/python_reproject_to_hips.py:66
    coadd_hips(['rgb_final_uncropped_hips', 'MUSTANG_12m_feather_noaxes_hips'], 'AshFigureWithACES')
  File ~/repos/python-reprojection/reproject/hips/core.py:327 in coadd_hips
    raise ValueError("tile_format values do not match: {tile_formats}")
ValueError: tile_format values do not match: {tile_formats}

...I think there's a formatting error there that I'll fix...

Comment thread reproject/hips/core.py Outdated
hips_order = [p["hips_order"] for p in all_properties]

if len(set(tile_formats)) > 1:
raise ValueError("tile_format values do not match: {tile_formats}")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
raise ValueError("tile_format values do not match: {tile_formats}")
raise ValueError(f"tile_format values do not match: {tile_formats}")

@keflavich

Copy link
Copy Markdown
Contributor

ok yeah, this error message is more useful, but strangely incorrect now...

Traceback (most recent call last):
  File ~/repos/avm_images/python_reproject_to_hips.py:66
    coadd_hips(['rgb_final_uncropped_hips', 'MUSTANG_12m_feather_noaxes_hips'], 'AshFigureWithACES')
  File ~/repos/python-reprojection/reproject/hips/core.py:327 in coadd_hips
    raise ValueError(f"tile_format values do not match: {tile_formats}")
ValueError: tile_format values do not match: ['png', 'jpeg']

@keflavich

Copy link
Copy Markdown
Contributor

ok, forced both things into .png format, and unfortunately I only get one image

@keflavich

Copy link
Copy Markdown
Contributor

Currently if there is overlap, the last image is shown on top. We could have some more fancy blending in future.

No, the first image is shown on top. 🤷

@keflavich

Copy link
Copy Markdown
Contributor

the tqdm progressbar doesn't work, it always reports 0%

@keflavich

Copy link
Copy Markdown
Contributor

@keflavich

Copy link
Copy Markdown
Contributor

looking at some of the index.html's generated by aladin.jar, getting the image to be oriented in galactic (not just projected into galactic, but also shown that way) requires non-negligible javascript in the index

@keflavich

keflavich commented Jun 27, 2025

Copy link
Copy Markdown
Contributor

the tqdm progressbar doesn't work, it always reports 0%

user error; I copied the code snippet above but I wasn't in a notebook

@astrofrog astrofrog marked this pull request as ready for review July 3, 2025 16:18
@astrofrog astrofrog changed the title WIP: Support for HiPS Add support for generating HiPS Jul 3, 2025
@astrofrog astrofrog changed the title Add support for generating HiPS Add support for generating HiPS and reprojecting RGB images Jul 4, 2025
@astrofrog astrofrog changed the title Add support for generating HiPS and reprojecting RGB images Add support for generating HiPS and reprojecting RGB images with AVM metadata Jul 4, 2025
@astrofrog

Copy link
Copy Markdown
Member Author

@astrofrog astrofrog merged commit 204bfd9 into astropy:main Jul 4, 2025
20 of 22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants