| Would you be interested in contributing towards an AB all sky survey? | |
|---|---|
| No. I wouldn't find such a survey useful. | |
| No. Satisfactory data already exists for me elsewhere. | |
| No. I would find such a survey useful, but I don't have the time, location or equipment to contribute. | |
| Yes, I would be interested in taking part. One or two fields maximum. | |
| Yes, I would be interested in taking part. Prepared to do multiple fields. | |
| Login to vote and view results. |
16.52
#...
·
 |
|---|
Ben: Ben, great points. If we were to go for a 12x12 deg mosaic for each field, then you would need to mosaic around 30 panes. However, we may decide to have higher spatial resolution in some areas. Virgo, Fornax etc. As to passband, I only suggest OSC (or RGB) as a starting point. Largely driven what people may have, particularly when using with a wide-field lens e.g. Sigma Art 135mm. Lets keep the idea of a NB all-sky survey also open. If we get enough people... Many thanks Brian |
|
16.52
#...
·
|
|---|
|
Great responses so far. Three days in an we have 12 volunteers for 1-2 fields and 14 for multiple fields. 90% of responders found find such a survey useful (although there may be a selection effect there). Optimistically that puts us around 25% of the way there. I don't know the half life of an AB forum post, but I will let this run for a further 4 days (a week in total) and see where we are. Equally as pleasing are the offers of help for the planning and curation phases even from those who may not be able to take part in the imaging. As far as I am concerned every who takes part in this survey has equal credit. Keep the comments coming. Thanks Brian |
3.81
#...
·
·
1
like
|
|---|
|
All, I'm wondering about the basic process on how this is all collected. I have an extreme what if i'd love to throw out there... What if there was a hosted service for storing raw and calibration images. You could allow a simple opt-in step for 'sky-survey usage'. It could allow you to easily share raw data with other users with a click or 2. I'm wondering if you could even get to the point to include pre-processing in an automated wizard. (imagine a wizard type workflow that include pre-defined standards) Maybe even basic post-processing workflow?? In a crude way, this is sort of what is required anyways, correct? We'll all need to compile images together somewhere, stitch them or register them to some degree, then make the finished data available. Everyone on this thread can certainly handle their own in processing data, so there's less of a draw. There's probably a ton more contribution even at higher focal lengths that could be included. Thoughts? totally absurd? |
|
16.52
#...
·
|
|---|
Nick Grundy: Hi Nick Not absurd. But totally beyond me to set up. And I worry a little that it violates the KISS principle. I had in mind simply to specify a product from each person i.e. a linear, calibrated image centred on a given field centre, with extent X * Y degrees and a pixel scale of Z arcsec/pix to a given depth. And a pipeline to reduce. I think that most taking part would be happy to be left to their own devices to generate such a product, consistent with those guidelines. And it shares the load as much as possible. Of course this begs the question of do we register/calibrate between fields. Iniitally, and following the KISS principle again, I had throught simply to have individual fields. But no doubt someone smarter than me could work out a way to do this. Indeed it may be easy if the calibration pipeline in rigorously followed. Hi Nick Not absurd. But totally beyond me to set up. And I worry a little that it violates the KISS principle. I had in mind simply to specify a product from each person i.e. a linear, calibrated image centred on a given field centre, with extent X * Y degrees and a pixel scale of Z arcsec/pix to a given depth. And a pipeline to reduce. I think that most taking part would be happy to be left to their own devices to generate such a product, consistent with those guidelines. And it shares the load as much as possible. Of course this begs the question of do we register/calibrate between fields. Iniitally, and following the KISS principle again, I had throught simply to have individual fields. But no doubt someone smarter than me could work out a way to do this. Indeed it may be easy if the calibration pipeline in rigorously followed. [I an very impressed by the new WBPP/SPCC in PI to give uniform data that can be registerd and mosaiced with out seams. Not sure we need access to the raw data for that - but happy to be proved long. A more comprehensive approach would also allow different focal lengths and passbands, but for the initial survey at least, I thought sticking with a single spatial resolution, colour (OSC/RGB) , field size and depth would be complicated enough. I agree that we should design anything like this out of the survey from the start. Of course, all this might just be my fear of IT, and this is all really simple to set up and will make our lives 10x easier. If so, bring it on! Brian |
0.00
#...
·
·
1
like
|
|---|
|
I've put together a widefield setup that is 5 Canon 6Ds and Samyang 135/2 lenses in parallel on a single mount. So basically the exact 10arcsec/pix you describe Brian. I love stitching together mosaics and have done way too much trial and error getting a scalable and repeatable workflow. I have many unpublished images in the works currently actually hahaha It goes without saying that this project is right up my alley, and I'd be keen to help! I have been thinking about reaching out to somebody in the northern hemisphere so I can get the whole galactic plane in a single image, so it makes sense that I try and get involved. I've also been thinking about the ideal exposure time and overlap per panel for such a project. Independently I was thinking 2 hours per panel from true dark skies (or longer equivalent from worse skies), at f/2. People can always be free to add move exposures to areas of particular interest. At this pace, I could conceivably shot 25 panels in a single (winter) night with my setup. So I only need 16 nights for the whole sky with unlimited travel time haha. The ideal overlap depends on whether the imager can accurately rotate each panel (and if not how far they are imaging from the celestial poles). Generally I find the more the better to help the stitching, usually around 25% minimum overlap My workflow starts out like yours, but I don't do BlurX per panel, as I've found it can create aliasing issues if the mosaic registration has to rotate the panels by small angles. I also use APP to integrate/merge the panels, but never to register them in APP, I use mosaic by coordinates in PIX for that. APP is great at registering a few panels, but as the panel count increases (say 20+) it can really start struggling, and its "black box" nature can become a real hassle. Agree on the standardisation of the processing. My own [linear] workflow for wide-field mosaics is |
|
16.52
#...
·
|
|---|
|
Wow DL. It would be great to have you on board. Both for your technology and experience! Wonderdul stuff. I havent tried APP to merge more than about 25 panels - see my lastest image which is about 21). I will admit it is a bit creaky and appears to put in a lot of green?? I must try your suggestion of mosaic by coordinates in PI. Last time I wasn’t successful, but with the new WBPP and SPCC it may work better. Also agnostic on the use of BXT - at this resolution itis debatable whether it is needed, but it does help to tidy up (a little) images off-axis. I haven’t come across the aliasing problems you mention, but that may be because I dont do as large mosaics or I have had a very light touch on BXT (star sharpness 0.1) I also note the individual fields wont require too much mosacing; 2 panes for a 135mm system and 4 for a 200mm system like mine. I did initally scipe around the Sigma 135mm lens as it appears to be quite popular here… Hiwever, the more comments I receive the more I think we might want to go for a full mosaic of the night sky properly curated online. I would know where to start, but there seems to be a few folks out there who might want to have a shot at doing it! Looking foward to working with you on this one and thrashing out details of the pipeline observing strategy (also think that 2-3 hours is about right. It is a little less than I gave my last mosaic around 3-4 hours per panel with a 200mm f/2 syatem in Bortle 2 - but that goes probably deeper than needed for this atlas. CS Brian |
10.80
#...
·
·
2
likes
|
|---|
|
Hey Brian are you familiar with the work of Nick Risinger who created the Sky guide App? it is getting a bit old now but he took a mosaic of the entire night sky which is over 600 panels from both north and south hemisphere. Really interesting story. he gives a brief overview of how he did it here which might be of some interest: http://skysurvey.org/blog/2011/2/17/for-all-the-nights-stars here is a picture of the rig he used: https://skyandtelescope.org/sky-and-telescope-magazine/nick-risinger-on-the-photopic-sky-survey/ also had a brief encounter with @John Gleason on Facebook the other day who has been working for years on an insane 1000 panel Ha mosaic with his FSQ 106…. Maybe there is room for collaboration…? |
0.00
#...
·
·
1
like
|
|---|
|
I will try to catch up with some of the messages that came over the night. First, at least for me, imaging near the polar region was no issue at all until now. Recently, I made an image of the IFN around Polaris. Yes, dithering (especially in dec) takes a lot longer, but that is acceptable. Besides from that, there should be no problems, assuming a good polar alignment. I myself have never done a mosaic before, so my knowledge about this part of processing is rather limited. But I was often amazed by the registering capabilities of PixInsight, and I am confident that there is a solution using PixInsight. If we can't make it work ourselves, I would bet that Juan from the PTeam would be glad to help us, considering the project. Regarding the necessary processing: We must define a reasonable, accurate and reproductive preprocessing pipeline that each image has to follow and additionally some form of postprocessing including color calibration and gradient removal. At least for me, gradient removal seems to become the biggest issue here. Color calibration is a purely scientific, measured approach, whereas with gradient removal, the user has to decide where there is pure background and where not. Another possible problem will be with very dense star fields along the Milky Way I guess. In my widefield images (50mm, so a lot more wide than with this survey), nearly all the nebula are buried within the stars. In my opinion, these are the two main processing issues we will have to solve. As far as my knowledge goes, PixInsight offers a script for mosaic generation that adopts gradients between different panels of the mosaic. Tools like BXT, NXT or other heavy processing that COULD degrade the data in the survey should be done after the combination of the panels, to ensure that it is done in a equal amount and is adequate for every part of the survey. Otherwise, the noise will be non-uniform or the noise reduction may be too high for some areas of the mosaic. CS Gerrit |
|
0.00
#...
·
·
2
likes
|
|---|
Brian Boyle: No worries, keen to be involved. With mosaic by coordinates, I've found good success with two things: Using the "advanced alignment" in image solver when generating an astronomical solution per panel, and having lots of RAM! If APP is registering the mosaics correctly though, there's no reason to reinvent the wheel right now. If you are looking to generate very wide fields, and perhaps even a full seamless 360 degree panorama, I think a combination of Pixinsight and maybe even PT GUI (once non linear) may be necessary. I'm definitely a big fan of BlurX, and use it on every one of my images. I just wait to use it on the joined mosaic, not the individual panels. |
3.87
#...
·
·
1
like
|
|---|
| For gradient removal, take a look at GraXpert. I often pass my integrations through it as a first step and then work in PI for most of the rest of the processing. It works very well. APP is fantastic and much easier tool to use for mosaics and they have worked on some very big projects. Stacking and data integration is their specialty. |
1.81
#...
·
·
1
like
|
|---|
|
A slightly different yet complementary idea: existing images on AB could be selected and placed on a Stellarium-like viewing interface. There could even be LRVB / SHO layers. I would bet @Salvatore Iovene has thought about it / worked on the idea. This would be an incredibly useful tool to look for ideas of future imaging projects. +1 for https://www.mdwskysurvey.org/ @John Gleason ’s gallery has some incredible Southern Hemisphere mosaics too. |
 #...
·
·
2
likes
|
|---|
A slightly different yet complementary idea: existing images on AB could be selected and placed on a Stellarium-like application. There could even be LRVB / SHO layers. Not a survey, but related: https://www.cloudynights.com/topic/876564-catalog-of-astrobin-image-of-the-day-for-cartes-du-ciel/ |
|
1.81
#...
·
·
1
like
|
|---|
| Great stuff and even better would be to see it right here on AB! |
|
10.80
#...
·
|
|---|
Great stuff and even better would be to see it right here on AB! Yes imagine a Stellarium style planetarium built into astrobin with its current search engine capabilities, that would be phenomenal! I’m sure a phenomenal amount of work for Salvatore too…. |
|
0.00
#...
·
|
|---|
|
Regarding GraXPert, modeling a gradient is not the complicated problem here, that can be done with a lot of different, relatively easy software. The real problem is to decide whether something is a wanted or unwanted gradient and where true background can be found. Imagine a 135mm deep image of a region in the swan or cepheus. If you have enough integration time, it becomes effectively impossible to find even a single pixel without some visible nebula. Even the best gradient modeling software will not solve this issue, as there is simply a lack of background data. I struggle with that increasingly often as I create more deep images with wide FOVs. As of now, I don't know of a sufficient, general and accurate solution for this problem. But with such huge projects, it will definitely become important. Assuming we actually go along with the survey, I would contact the guys from the mdw survey to ask for advice on how they have done it, unless we can figure out a good solution. Their images seem to be completely gradient free. CS Gerrit |
4.20
#...
·
·
1
like
|
|---|
|
The idea is very cool and I encourage everyone capable of helping to make it happen. I would have participated, but I only have a 24-70mm F2.8 with an unmodded DSLM. No tracker, no mount yet. Bortle 5-ish to 6 sky. I went for the 3rd option. |
|
16.52
#...
·
·
2
likes
|
|---|
| @emmanuel_valin@Steeve Body @Salvatore Iovene What a wonderful idea for curation/presentation of any survey. Well beyond my capabilities of course, and I hadn't dared to think such a thing could be possible. I would be have been happy just securing 400 10 x 10 degrees separate but overlapping fields being made available. Of course, the effort required to put something together may simply make this impractical - I don't know. Three days in and we now have 40 volunteers for imaging. This is almost half way there. And a similar number interested in the survey, who may be able to help in other ways. When I first proposed this as a survey, I was drawing on Astrobin's power as an international community. If we can further use that power to cover all the technology bases required, then then would be just as amazing. Very shortly I will start a separate thread (this one is now getting quite long) where we can start to discuss survey principles, design, personnel, tasks and timescales. [Of course, I could simply keep this thread going if people feel that is better] We have five major components of the survey. At least that I can think of - most likely there are more. 0. Survey need 1. Survey field centre generation, distribution and progress monitoring 2. Survey field imaging and processing 3. Survey curation and field combination. 4. Survey publicity This thread was started to assess feasibility of resourcing 2, and the threshold requirement 0. But along the way we have had volunteers for 1 and a great discussion around 3. We haven't really discussed 4. With all the great input and enthusiasm, the Astrobin Community All sky survey looks like is it approaching the starting line. I will be in touch soon.... Thank you Brian |
|
16.52
#...
·
·
1
like
|
|---|
The idea is very cool and I encourage everyone capable of helping to make it happen. Thanks for your support. Even if one can't contribute to the imaging part, there are many other survey aspects that need people (see my message above). An important principle of the survey should be inclusivity - those that want to and can contribute should be welcomedin. Of course, we also have to manage this collaboration, and it is likely that each survey task area would require a manager so we can manage the "controlled anarchy" - a term that someone once described the best research projects as being. CS Brian |
|
4.20
#...
·
|
|---|
| @Brian Boyle Well, I don't know much of anything, only a basic knowledge of PS (not that that might help in anything). If I could somehow help in any way, I'll try. CS to you. |
0.90
#...
·
·
1
like
|
|---|
|
Having recently completed a 90 pane (almost) full Milky Way mosaic at 85mm focal length which took me 11 months I can say that Astro Pixel Processor was right on the edge of its capabilities and I spent about 2.5 weeks trying to overcome performance issues and bizarre registration problems stitching it all together. The developer has admitted in the APP forums he is not putting any effort at this stage into fixing the large mosaic problems. I would suggest doing a proof-of-concept in the early stage of the project - say a single frame (or a few) from all contributors - and making sure that such a large mosaic can be created with relative ease. Alistair Symon would be another good resource with his highly detailed Milky Way mosaic with 2500 hours of imaging. Oh and I would be happy to contribute to the project  |
1.20
#...
·
·
1
like
|
|---|
|
I couldn't resist having a play at the tiling question. I went for the simplest option of keeping the centres of the field edges parallel to lines of RA/DEC and have assumed 10 x 10 degree fields. If we go for a 20% overlap, then the cleanest solution I can find has DEC centres at 86°, 79°, 72° [... every 8° ...] -72°, -79°, -86°. The number of fields for each declination sweep ranges from 4 at +/- 86° to 45 at the equator. The total number of fields required is 662. As each new field adds a unique 64 (= 8 x 8) square degrees of coverage, this compares pretty well to the theoretical minimum of 41253/64 = 645. With a 10% overlap, the DEC centres are 86°, 79°, 72° [... every 9° ...] -72°, -79°, -86°. The number of fields for each declination sweep now ranges from 4 close to the pole to 40 at the equator. The total number of fields is 532 versus a theoretical minimum of 510. The image below illustrates the 20% coverage option. The N pole is approximately at the top and the red line marks the equator,  |
|
1.20
#...
·
·
3
likes
|
|---|
|
Regarding stitching the fields together (and noting the discussions above about the limitations of APP and PI with extremely large mosaics), I wonder if another approach might be worth considering. From what I've read about PI's advanced plate solving, it can generate astrometric solutions accurate to about 0.1", much better than the 10" pixel size being proposed for this project. Assuming that users can access the plate solution across the field of the image, this could be used to interpolate a new image on a regular and absolutely defined 10" x 10" RA/DEC grid. Assembly of these resampled images into the master image would then be straightforward - they just get dropped into the correct location based on their now known exact coordinates. I think this approach would have a couple of advantages. First, it means that plate solving and resampling can be done on the scale of an individual field image (3600 x 3600 pixels for a 10 x 10 degree field and 10" x 10" per pixel). This should be well within the capacity of a standard PC. Second, any field distortion and image scale variation is corrected out, which should simplify alignment of neighbouring images potentially taken by different observers with different equipment. For simplicity, I suggest that the compiled master image is built on a rectangular grid which would measure 360 x 360 = 129600 pixels in RA by 180 x 360 = 64800 pixels in DEC. Of course, the image would be oversampled as one approaches the poles, but the wastage is fairly modest. The total image size (8.4 billion pixels) is about 57% larger than the number of 10 x 10" pixels that cover the sky (41253*360*360 = 5.3 billion). The advantage would be simpler assembly of the master image and simpler reconstruction of an arbitrary field requested by an end user. |
|
16.52
#...
·
|
|---|
| . |
|
16.52
#...
·
|
|---|
| . |
|
16.52
#...
·
|
|---|
| . |
