Ed Wiley suggested that this new topic be created, because there was a lot of general discussion about CMOS cameras in his post from last year "Call for Action: CMOS Photometry", in which Ed suggested that it would be useful to have a manual or guide for photometry with CMOS cameras.
The last post in "Call for Action: CMOS Photometry" prior to Ed's suggestion to create this new topic, concerned gain settings and binning in CMOS cameras.
I copied that post into a document which I've attached. No-one (as of a few minutes ago) has yet posted a resonse to it.
Hello Roy, While I can't respond directly to your post, I'd like to say that I am planning to do CMOS photometry with my new homebuilt 32cm F/2.8 reflector. I plan to use the QHY5III Color CMOS camera at prime focus with an Astrotech Coma Corrector.
Would appreciate any comments on this setup for VSO from the many experts here.
Some specs on this camera are as follows:
- CMOS Sensor: Sony IMX178 ExmorR CMOS color sensor
- Effective Pixels: 3072 x 2048 (6 MP)
- Effective Area: 7.37 mm x 4.91 mm
- Pixel Size: 2.4 µm x 2.4 µm
- Readout Type: Progressive Scan
- Full Well Capacity: 15ke-
- Readout Noise: 2.4e - 0.9e
- Anti-amplight Control: Yes
- ADC Sample Depth: 10/14bit; Output 8bit/14bit
- Pixel Binning: 1x1, 2x2
- Optic Window: AR+AR
- Support Software: ASCOM, SharpCAP
My thanks! This it the right approach to getting all the thoughts about CMOS cameras for photometry in one place under a thread title that attracts attention. I have been inspired and am selling some stuff off so that I can get a CMOS mono (an ASI 183 Pro cooled) and join the crowd as we march towards the CMOS universe of photometry.
My hope: Discussion of general principles of CMOS photometry stated in such a manner that clueless people like me can successfully obtain excellent photometry with these cameras. Ray, your word doc is a great start. I would urge all to stick to this goal, it’s easy to get diverted.
Among topics I am interested in (there must be others).
1. Dynamic range, gain setting and offsets. How do I achieve the greatest dynamic range with a 12-bit camera?
2. The effects of off-camera binning and its relationship to #1. (From what I can see, off-camera binning is fine, but how much?)
3. Matching cameras to OTAs in the age of CMOS cameras with tiny pixels (for those who cannot now afford Keplers and await less expensive cameras with larger pixels). For example, I plan on using the 183 with an 80mm refractor for bright variables. How about my C11 edge at 1950mm FL? Again, related to #1.
I know these topics have been addressed, piece-meal. But if you informed CMOS photometrists will use this forum to gather them together a guide might emerge.
The ASI183MM-Pro, which I own, is a 12-bit camera. The ASI178MM-Pro, which has the same pixel size, full well capacity, similar read noise, quantum efficiency, is a 14-bit (though the 183 has 20MP resolution vs the 6.3MP of the 178). Unfortunately, ZWO no longer sells the cooled version of that camera. However, QHY does sell a camera with the same sensor.
Please see the document attached to the first post in this forum, 26 Jan 2020. This post is a response to some of the quetions posed in that document.
My experience is limited to the ZWO ASI1600MM cooled monochromatic CMOS camera. It allows binning, and yes, various gain settings can be used, both with binned and unbinned images. The range of gain settings is from about 0.2 electrons/ADU to 5 electrons/ADU. My understanding (and I am not a CMOS expert, merely a user) is that binning in this camera (and at least some other CMOS cameras) is done with software after the pixels are read out.
The camera has 12 bit ADC. Note that the line profiles in the attached PowerPoint (which I'll explain) show maximum counts above 60,000. For 12 bit ADC, the maximum count per pixel should be 4096. The ASI 1600MM software, in RAW16 mode, multiplies the actual counts by 16 to give the numbers you see. Curiously and enigmatically, in both unbinned and binned 2x2 mode, the sensor saturates at 65,504 counts per pixel, as seen after taking a series of flats at various exposures for linearity testing, and AstroimageJ to analyse the FITS images.
See attached graphs of the camera's stated performance from the official manual.
The attached PowerPoint is somewhat complicated. It shows the field of FR Cet, a 6th mag variable of uncertain type, and non-transformed V mag photometry of the var and the check star. The only reasonable comp/check stars in the field with my 'scope are 8th mag. The 7th mag star is a spectroscope binary, and is variable, although variability has not been reported to my knowledge.
Now, to get high enough counts for reasonable photometric precision without saturating the sensor, it is necessary to increase the exosure, which can be done after 'increasing' the gain from unity to 5 electrons/ADU with my scope and the exposures I used. The dynamic range of the sensor increases as the gain settings are increased up to 5 electrons/ADU. The PowerPoint shows the results for the FR Cet field, and the line profiles of the var and check stars at the various gain/exposure settings. Should be self explanatory on careful study.
I should have emphasized that tuning the exposure, gain setting and degree of defocus of the image, with the aim of getting the maximum counts per pixel, in the linear range, for the brightest star measured, should, in my experience, optimize the precision.
Some "nitpicking" on the wording (this might be confusing when writing it all up for end-users);
I guess most people would read "increase the gain" as going to LOWER values of electons/ADU , as the more natural unit to measure "gain" is actually the inverse ADUs/electrons. So 5 electrons per ADU is actually a "lower" gain setting than unity gain , which is consistent with specifying gain in logarithmic scales like dB, or ISO, or 0.1 x dB as on
I agree. That's why I put 'increase' in quotes: increasing the number of electrons per ADU.
I should have explained: the photometry in the PowerPoint attached to the previous post was taken during the bushfires (you calll them 'wildfires') in eastern Australia, so the photometry is suboptimal from smoke haze. Also, the effects of increasing the electrons/ADU can be seen best just looking at the photometry of the check star. The variable is enigmatic, hence the reason I started to study it.
Furthermore, the pre-meridian flip data for the variable at 2.75 e/ADU was saturated. Adjustment was made during the meridian flip, hence two line profiles for the 2.75 e/ADU setting - one before and one after the flip.
No-one has responded to Ed's post of 26 January. His final sentences were: "I know these topics have been addressed, piece-meal. But if you informed CMOS photometrists will use this forum to gather them together a guide might emerge."
Ed, could I suggest, since it was your idea initially, that you approach HQ formally with a request that the AAQ Board consider setting up a group (presumably of volunteers, with representation from HQ?) to plan and implement the drafting of a CMOS Photometry Guide.
I worry about carts before horses here. There is a big difference between (1) having general, well-established CMOS Photometry Best Practices on which to base a CMOS Photometry Guide, and (2) writing that Guide.
So: do we have comprehensive, agreed-upon CMOS Photometry Best Practices? Without them, I'm having trouble seeing the point of attempting to write a Guide until they are agreed. Perhaps the first step is to get specific agreement between CMOS Photometry leaders on what those Best Practices are.
Eric, I can see your point of view. Before I bought my ASI1600 camera last year, I scoured the internet for information on the use of CMOS cameras for photometry. I found very little definitive about the practitcal matters I was looking for - apart from testing for linearity!
I think it is reasonable to try to determine what is best practice for CMOS photometry, or to write a CMOS photometry guide (the latter word was chosen deliberately), however you wish to describe the aim. Within the AAVSO, I can't think of a better way than to have a group of people who have experience in photometric studies of variable stars come together to pool their expertise and experiences.
I believe something like this is likely to be more successful if it has in principle support from the AAVSO Board, and a formal structure.
However, that's only my opinion. If there is another way to arrive successfully at a description of best practice for CMOS photometry I'd be happy to support it.
I have a question about CMOS, binning and "virtual" bit depth. Someone said that, because CMOS bin in the software, a 2x2 binning will give a 12-bit camera a virtual 14-bit. At least for some CMOS software, the ADU count for each pixel is averaged when they're binned. If four pixels have an ADU count of 3000, 2000, 2000, and 1000 (assuming the Gain is set so the ADU count at full well capacity is 4095), the ADU count of the 2x2 binned pixels would be 2000. It just looks like a loss of data along with running the risk of one of the binned pixels being saturated and not knowning it rather than getting an extra 2 bits of bit depth. The only way I see going from 12-bits to a virtual 14-bits is by adding the ADU count of the binned pixels together to get 8000 ADU with the total saturated ADU count of the four pixels being 16,380 (4095 x 4 or 12-bits x 4 = 14-bits) before being multiplied by 4 to be converted to a .FIT file (14-bit x 4=16-bit). Or better yet, in the case of a 14-bit CMOS camera, just add the ADU count fo the four pixels and storing the data for the binned pixels as a .FIT file without multiplying the ADU count to fit 16-bits.
Rather than producing yet another Photometry Guide, why not just take a step back, look at the existing CCD Guide, and, in a first step, mark up all the bits and pieces in it that would have to be qualified / adjusted/ addded-to in order to have a next edition "CCD & CMOS Photometry Guide".
I'm not convinced at all that the differences would justify spawning off a whole separate document (with all the follow-up work associated with it like translations (!)). It would be much easier to translate only the changes than to make a whole new document, with a lot of replication, and then have to translate that.
IF HQ could somehow export the current CCD guide to a shareable Google Docs document this could even be a distributed, collaborative effort.
I'd support that. Since the CCD observersing manual is an official AAVSO publication, the idea will, I presume, need to be endorsed by the Board. I still believe that the best way forward would be for the Board to set up a formal process with a group of experienced observers, someone to lead the group and perhaps HQ representation.
Someone would still need to place a formal request with the Board to get the ball rolling. Ed?
With the ZWO ASI1600MM, as far as I can see, binning simply averages the count across the binned pixels. Thus, at unity gain, saturation is 65,504 counts both for unbinned and binned 2x2 images. I think that the idea of data being lost, or the reisk of one of the pixels being saturated is not anything to worry about. Testing provides data on linear range, and line profiles and other statistics provide information on the pixels containing signal and the surrounding sky. It would be apparent if there is any saturation.
The suggestion that precision might be lost in the situation under discussion (software binning in CMOS cameras where the binned pixel counts are simply averaged) is, IMHO, purely speculative, and I suspect incorrect. It can be tested, and my preliminary tests support my opinion.
A CMOS Guide is already under discussion by a small group involved with the other guides and Choice courses. Since a lot of common info is present in these guides, it is felt that it would make most sense to use them as a basis for a CMOS guide which would add the technical details and different procedures needed to properly conduct CMOS photometry.
IMHO, it is not really necessary to get some approval from the Board BUT to gather a group which is willing to take on this effort and present it to them when the guide achieves a critical mass. As Ed was trying to indicate, there are details that we suspect/know some members already are dealing with. Their input would be helpful to define what these technical/procedural details are. In fact, two BSM telescopes now utilize a ZWO ASI183 camera for photometry with good success. In the near future you will hear about the technical details of this work BUT it would be useful to hear from others about their procedures and results.
So, readers of this post should keep the faith and trust that such a guide is under consideration. And understand that the help of current CMOS users is needed/appreciated. So far no one has provided specific details about how they operate their CMOS cameras with respect to gain, binning, etc? Stand up and provide your comments. Don't worry about being uncertain or incorrect. Tell us how you run your CMOS camera. Let others poke holes in the technique OR support and confirm it. We can learn from each other. Describe your techniques, your settings, your results. Accept any critique or disbelief that may occur. Learn from it! Help the few people who are discussing the guide to develop the necessary methodology sections.
With this information, I know that there at least three of us who are prepared to take on the effort of writing a CMOS manual, and a professional to critique the document.
Thanks Ken. Good news. I'll put something down in a MS Word document about how I use the ASI1600 cooled mono camera, and attach it to a post in this forum.
The attached is a short first draft, summarising my experience over a few months with this camera.
I can attach examples of every point that I make: screen shots of the camera control dialog, graphs of performance from the camera manual, linearity plots, light curves, averages and SDs of results, screen shots of images binned and unbinned, line profiles. But I thought I would get the words on the forum quickly so people could comment, critique or ask questions as wish.
Thank you very much, Roy.
I find your report very precise and useful. I'm eager to read your report on the photometric precision of the camera.
You know, this camera could be perfect for a 300 mm refractor that I am setting up..
I am seriously considering to purchase it. One question: given the tiny minimum exposure, it is clear that there is no mechanical shutter, so I suppose that to take darks you need to use a black filter?
I think that this is a very productive discussion and we appreciate all of you who are using CMOS cameras helping those of you who are learning to use them properly.
As for the question of whether to write a new CMOS manual versus simply updating the CCD Photometry Guide to include CMOS and special notes on where things diverge, we at HQ have no strong feelings about this one way or the other, we just want what's best for the community.
In case it is decided that modifications to the exisiting CCD Photometry Manual would help (at least temporarily), I have created a Google Doc out of it, and am happy to share the link with whomever would like to be involved with the project of modifying it. I apologize in advance for the fact that not everything converted from InDesign to a Google Doc perfectly, especially the images and page breaks, but feel free to fix it. I am definitely no expert on formatting things in Word!
Contact me for the link: sara [at] aavso.org
All the best,
My last post on this camera (30 January) had an attached 3 page document which was text only with no illustrations. The attached 14 page document is much more fleshed out, and also includes screen shots, images of photometry fields, and other figures and tables of results.
For those wanting to cut to the chase, there are two new sections at the end: BV transformation coefficients on pages 12 and 13, and transformed BV photometry results using those coefficients on pages 13 and 14.
Apologies for any typos, and for formatting not consistent throughout.
The document does not really set out to be a users guide, and focusses mainly on results of testing. It does, however, describe how I use the camera, and some of its peculiarities.
Roy, your 3-page document was already very useful. Of course, this is much better, and it includes a photometric precision evaluation which I find very valuable.
PS: one question about Fig. 13. According to the time scale, it seems to me that the two light curves do not refer to two consecutive night but to two consecutive runs in the same night. Is it so?
my first test at a friends home with a 10 cm Apo f/7 refractor and the QHY600 on a HADS (High Amplitude Delta Scuti) star, V0451 Dra. It works like a charm for photometry. So far only 2x2 binning is possible with the driver and MAXIM DL. The V0351 Dra varied between mag 12.5 and 13.1. I used no filter as I do not yet have a connection with the FLI filterwheel. With 15 s exposure the uncertainty using for Ref the 10.48 star and for check the 11.43 star from the AAVSO sequence was between 0.015 to 0.02. No much time was invested into focussing. With 30 sec exposure the uncertainty dropped to 0.009 to 0.013. It was very humid, but clear with bright moon. File size is 30 MB. Another friend will write a PYthon program to do 2x2 binning on the images. We also tried to image the asteroid (33165) joschhambsch. It is about mag 18.9. MY friend succeeded with the double of the FL (about 1500mm) and a 20 cm refractor and ST10 to get it on images taken with subs of 2 min and in total 56 min exposure. But on the small refractor images due to a large FWHM I had (about 8 arcsec) I could not really identify the asteroid.
So for a first test I am happy. very smooth images and nearly no noise running the camera at at -10 deg.
Forgot to mention no amp glow at all.Only for the 2 min subs I took 8 darks and calibrated the images.
No calibration done for the HADS images (30s and 15s exposures).
I've read about concerns of pixel to pixel sensitivity that may affect CMOS photometry. Is this an issue with the current generation of CMOS chips? Best regards.
Just a quick note. Inspired by the discussion I have acquired a ASI183mm Pro camera. I already have an Atik EFW2 filter wheel that should work. I look forward to learning how to use this camera and specifically picked it because the price break is likely to encourage new photometrists. Thanks to all of you for your productive posts, I have learned a lot and look forward to learning more. No need to reply and gum up the thread.
Good measures, Ed
I had some time to think about binning and saturation, and I would like to ask if this is a good idea. If anyone is going to bin with a CMOS in software, then they should take a test image at 1x1 or not accept anything higher than the maximum linear ADU value divided by the number of pixels being binned. Hypothetically, if someone was to take images of a star at 2x2 binning in which the binned pixels have a reading of 65535, 40000, 40000, and 30000, the average ADU count would be 43,883.75. While the average is still below saturation and below the maximum value for linearity for most cameras, that one pixel would give a false reading and make the star appear dimmer that it really is and the person taking the image would have no way of knowing without taking an unbinned image prior to taking the binned images.
Also, about ZWO cameras and VPHOT: ZWO camera software records exposure time in microseconds. So anyone using a ZWO camera that wants to submit to VPHOT will have to run the .FIT files through software for editing FIT headers first or run it through third party software I know that from experience.
If you are going to speculate on the effects of binning and do hypothetical math, the figures need to have some relationship to reality. To suggest that four adjacent pixels would have ADUs of 65535, 40000, 40000 and 30000 is not realistic. The attached defocussed unbinned image of an 8th magnitude star is from my ZWO ASI1600MM camera. The peak value is 62225. To get to 40000, you need to move about 11 pixels across the line profile in one direction, and even more in the other direction.
The real test of what works and what pitfalls to be aware of come from actually using the camera and testing the results under various conditions. I have carried out linearity tests on unbinned images and images binned 2x2, and performed photometry on both unbinned and binnned images.
In practice, there is no problem if you take care not to go too close to the top of the range of linearity.
That would depend on how much the scope is defocus, target star brightness, and exposure time. The last star I submitted to VPHOT was an 11.6 mag. Also, I try to keep the FWHM at 3-5 pixels. The FWHM of RS Gru from your ASI1600 was around 26-28 pixels. And IIRC, if the image is defocused too much, then sky brightness would create too much noise in the image. I don't know if I would want to defocus an 11 mag star by a FWHM of 26 pixels.
12 Bit only have 4096 ADU, never 62225. This would be over exposed. I would buy a 14 bit or better a 16 bit cmos camera.
2^12 = 4096 bits or ADU
2^14 =16384 bits or ADU
2^16 = 65536 bits or ADU
So more bits, can store more light at/on the pixels, before they are saturated.
If the pricing of the CMOS camera is getting near >2000 $, a Moravian CCD comes close to the point of interest. ~2500,...3000 €,
Within the www.BAV-astro.de, the Moravian G2 1600 Mark II, is recomended.
It has: 9µm pixelsize, 16 Bit, and a Full well capacity of 100 000 electrons, and important: no anti blooming gate, which is not so good for photometry. So with a narrow starfield, faint compare stars can be used with no problems, because they have enaugh light gathered.
Yes, of course, a 12 bit camera has only 4096 native ADUs per pixel. The ZWO ASI software multiplies this by 16, hence the counts for full wells are more than 60,000. This is -not- the same as a 16 bit camera.
I study brighter stars, so 12 bits are enough for me. I have worked out how to optimise the precision.
Here's a question: couldn't the lower bit depth be compensated for by spreading the light from the target and comp stars across more pixels? Besides defocusing there are also CMOS cameras with smaller pixels.
A camera with either a KAF-0402 or KAF-1600 would have a camera with a pixel size of 9 µm. The IMX178 and IMX183 chips have a pixel size of 2.4 µm. A camera with a KAF-0402 or KAF-1600 chip does have 16-bits but the cover have over 14 times the area. So those 65535 ADUs is covering light falling on one 9 µm × 9 µm pixel while the IMX183 chip would have roughly fourteen 2.4 µm × 2.4 µm pixels with 4095 actual ADUs each assuming both chips are used in the same telescope with the same seeing conditions. And the IMX178 chip has the same pixel size as the IMX183 but it has a bit depth of 14 bits so that's fourteen 2.4 µm × 2.4 µm pixels with 16383 actual ADUs each.
Photometric images are taken with an 8-inch f/10 SCT with a 0.7 focal reducer with seeing conditions at 2.6 arc-seconds for FWHM. An SBIG camera with either a KAF-0402 or KAF-1600 would have a camera with a pixel size of 9 µm is used. The pixel angular resolution would be 1.3 arc-seconds so a FWHM at 2 pixels for that camera and telescope. Now replace that camera with either a ZWO ASI178MM or ASI183MM. The pixels are only 2.4 µm for both cameras. That would be an angular resolution of 0.348 arc-seconds per pixels and a FWHM of 7.48 pixels. IIRC, the recommended aperture radius is 1.5×FWHM. So the images taken with the CCD camera would have an aperture radius of 3 pixels. So that would be approximately 28 pixels in the aperature circle. For the camera with either CMOS chip, the aperature radius would be 12 after rounding up from 1.5×FWHM (1.5×7.48=11.22). So the light from the same star would be spread across 452 pixels as opposed to 28.
An analogy would be measuring heart rates. Human reflexes aren't fast enough to accurately click a stop watch for a single pulse. Rather, the number of pulses are counted over a duration of 15 or 30 seconds and multipled by 2 or 4, respectively. The indivdual pixels of most CMOS cameras lack the precision of a 16-bit CCD camera pixels but that could be compensated for by spreading the light from a star across more CMOS pixels and defocusing isn't necessary if the CMOS pixels are small enough.
If I am mistaken, let me know where I'm going wrong.
Compensating for CMOS cameras with low bit depth by spreading the light from each star across more pixels is, in my opinion, the correct way to use such instruments, and is supported by actual measurements, as posted in this Forum. Defocussing and even smaller pixel size would both do this. But even for the smallest pixels, defocussing and increasing exposure will improve S/N even further.
Be careful with the number of 'pixels in the aperture circle' as opposed to sampling based on pixels per FWHM. See the attached data and graph, taken from a well-focussed image on a night when the seeing was about 3.8 arc secs. The FWHMs of well-focussed non-saturated images of stars do not vary with magnitude, whereas the seeing disk, the actual image itself, does increase in size with increasing brightness. The attached shows the actual number of pixels in the seeing disks of stars of various magnitudes from one of my images.
It is intesesting to ponder these results when thinking about the usual definition of 'over-sampling', which of course is based on pixels per FWHM, not pixels in the seeing disk itself.
I can only describe what works for me with my 12bit camera and 80mm and 120mm refractor with relatively bright stars. I have been testing since May last year, have posted my methods and results here and elsewhere, and find good linearity, and precise and accurate photometry.
Your camera, 'scope and targets may be different. Methods or you may be different. So yes, it depends.
The following is a post that Arne made in the aavsonet forum in 12/01/2018. It relates to binning in a CCD camera but got into discussions of gain and binning in both CCD and CMOS cameras. I used it as a start for me to understand these parameters in a CMOS camera. I placed my thoughts in the subsequent section. Comments?
<<Binning involves several steps.
- the native pixel accumulates charge (converted photons). These pixels have a specific full well capacity depending on the sensor.
- during readout, each row is shifted to the serial readout register. The serial register "pixels" actually are bigger than the imaging pixels, and so typically have twice the full well capacity of an imaging pixel. This means you can bin by 2 in the vertical direction (transfer two rows into the serial register before readout) without losing any charge.
- then the serial register is transferred to the summing well. This well is like an even bigger pixel, typically with 4x imaging pixel capacity. This means it can hold a bin-by-2 row binning into the serial register, followed by a bin-by-2 column binning of the serial register, without losing any charge.
- finally, the summing well charge is amplified (gain) and sampled by the analog-to-digital (ADC) unit. If the gain is adjustable, it may be set so that 2x2 binning yields 1/4 of the gain of a 1x1 situation, so that the ADC does not overflow.
So the questions for any given sensor and camera are: (1) are these "pixel" capacities typical, so that 2x2 binning does not overflow the summing well? and (2) does the system adjust the amplification gain depending on the binning, or is there only one fixed gain for any situation?
If only one fixed gain, then you will overflow/saturate the ADC and you can only expose each native pixel to about 1/4 of its maximum capacity without causing saturation. If the gain is adjusted, then you might sample with 10 electrons per count rather than 2.5 electrons per count. While that permits exposing to the true full well depth of each native pixel, it means that a count for a binned image is different than a count for an unbinned image. Perhaps more important, since the gain changes, the bias frame may also change. You should take biases and darks for each binning situation.
Binning is ok, and entirely appropriate for oversampled cases such as small pixels on long focal length cameras. On OC61 for example, we bin the native 12-micron pixels into 24-micron pixels (even then, the scale is 0.55arcsec/pix!). On BSM_NH, for another example, we're using a ZWO 183 camera with 2.4 micron pixels, which yields 1.1 arcsec native resolution, below the Dawes Limit for these telescopes. So we bin 2x2, and defocus to yield 1.5-2.5 pixels per fwhm. Binning helps for the 183 in many ways, including making the 12-bit ADC into effectively a 14-bit ADC, because for a CMOS detector like the 183, the binning is done in software. I won't talk about CMOS again in this post.
Focal reducers are a different issue. Again, if you have a long focal length telescope, you may want to use a focal reducer to keep from oversampling with a given sensor. However, you are adding an optical element to the optical path. Focal reducers tend to only correct human-visible light; how well they perform form UV or NIR light is unknown. They also can "ghost" so that bright stars might have secondary images elsewhere in the field of view. They also can increase the vignetting. They are a reasonable solution to the oversampling case, but you need to be aware of their potential interference when doing photometry.
Regarding binning, what I highly recommend is to do a linearity test on your sensor. Usually regular CCD cameras like the SBIG line will have a straight line up to a certain count value, and then curve over, asymptotically approaching some value which may be considerably below the ADC maximum count. For an ST-7XME, for example, deviation from linearity may occur around 48K counts. If you see this kind of curve, then you are sampling to full well on your sensor, even in the 2x2 mode. If you instead see a straight line up to 65535, then most likely you are not sampling to full well on your sensor, and instead are being limited by the ADC. For most anti-blooming-gate (ABG) CCD sensors, because of their inherent nonlinear behavior near full well, vendors typically increase the gain so that the linearity curve is linear up to the full range of the ADC, and so this test for 2x2 gain doesn't work as well.
I hope this helps!
My thoughts below:
Note that this post by Arne includes a description of well depth and gain for both CCD cameras and CMOS cameras. It takes some effort to interpret how it determines how one should set gain for each type of camera on the basis of both binning and well depth. My interpretation is aided by an understanding that in CMOS cameras like the ZWO ASI183, larger binned pixels (e.g., 2x2) are NOT created on the CMOS camera. (They may be on a CCD camera.) Each native (single) pixel on a CMOS camera is always read out and amplified (gain) before it is converted to a digital count in the ADC chip and before it is summed with adjacent pixel counts IF binning is requested. IOW, the binning/summing of adjacent native pixels is simply an arithmetic addition of the counts in the software binned (e.g., 2x2) pixels.
So, initially one should adjust/set the gain (e-/ADU) so that it takes some number of detected electrons/photons impinging on that pixel to fill the well depth of the pixel when the ADU count reaches the bit depth (e.g., 2^12 bits) of the camera. IOW, if the well depth of the ASI183 camera is 15,000 electrons and the max adu count of the 12 bit camera is 4096 counts, the gain should be set at 3.66 (15000/4096). IOW, it takes about 4 photons/electrons to be collected in the pixel to yield one ADU count.
Interestingly, if one subsequently (in software) bins 4 pixels (2x2) into a larger pixel, the 4096 adu counts in the smaller native pixels get added together. So we have four times more adu counts (4096 x 4, or 2^12 x 2^2) in the software binned/added pixel. The 2^12 bit pixel/camera looks like a 2^14 bit pixel camera.
The binning of the CMOS camera thus increases the bit depth of the camera and can also be used to more closely match the final pixel size and the seeing according to the Nyquist rule (2-3 pixels per fwhm seeing).
The read noise of a CMOS camera (e.g., 1-3 e-, 2^1 bits) is generally smaller than that of a CCD camera (8-10 e-, 2^3 bits). This helps reduce the random noise which adds some random bits to the real adu count of a star. In this case, the CCD camera appears to have slightly less effective bit depth than a CMOS camera and they become more similar in regard to effective bit depth. The CCD still has more bit depth, perhaps 2^15 vs 2^14.
So read this hypothesis, pick holes in it. Tell us if that’s how you set your gain? Perhaps Arne will respond and tell me/us if this make sense at all?
Ken, here are my thoughts after reading yours:
"IOW, the binning/summing of adjacent native pixels is simply an arithmetic addition of the counts in the software binned (e.g., 2x2) pixels."
For the ZWO ASI1600MM CMOS (12 bit) camera I use, if I understand correctly, the count (ADUs) per pixel after binning is simply the average of the counts of the 4 pixels binned (in the case of 2x2 binning), not simply the sum of the counts from those pixels.
"IOW, if the well depth of the ASI183 camera is 15,000 electrons and the max ADU count of the 12 bit camera is 4096 counts, the gain should be set at 3.66 (15000/4096)."
But the gain setting is just one of the parameters to be tuned. Since we are talking about a 12 bit camera, it will (in my experience) be necessary to defocus the images, so degree of defocus is another parameter to be set. The third parameter is length of exposure. Thus, in my experience, one needs to image the brightest star of those being studied, take trial exposures (perhaps based on experience) then tune the exposure length, degree of defocus and gain setting until the ADU's per pixel are as close to the upper limit of the linear range as you can get, based on your own linearity testing. I would reduce the gain (increase the e/ADU) only for bright stars, where even with defocussing and short exposures, the exposures are (in practice) too short to give precision photometry. Re-setting the gain and thus being able to lengthen the exposures will under these particular circumstances increase the precision. (This purely empirical procedure is based on my own testing to optimize precision in a field where 6th and 8th mag stars are to be measured through a 120mm refractor. I ended up with a gain setting of 5 e/ADU, the highest that can be set in my camera).
"The binning of the CMOS camera thus increases the bit depth of the camera and can also be used to more closely match the final pixel size and the seeing according to the Nyquist rule (2-3 pixels per fwhm seeing)."
Sorry, can't comment on this. I have never tried to match the pixel size and seeing as described. I guess this is beause most of my photometry has until recently been with a DSLR camera, for which all images for photometry are defocussed.
"The read noise of a CMOS camera (e.g., 1-3 e-, 2^1 bits) is generally smaller than that of a CCD camera (8-10 e-, 2^3 bits). This helps reduce the random noise which adds some random bits to the real adu count of a star. In this case, the CCD camera appears to have slightly less effective bit depth than a CMOS camera and they become more similar in regard to effective bit depth. The CCD still has more bit depth, perhaps 2^15 vs 2^14."
I get out of my depth easily here. However, if I understand correctly (and it is possible that I do not), with dark skies (low background counts) and faint targets read noise can predominate over shot noise. In contrast, at an observing site near a city with bright background skies, shot noise predominates. My only point (valid or not, I don't know) is that your comment may need to be considered differently under the two circumstances.
I hope the above is relevant enough to be useful. My other hope is that it is not misleading.
I think the first setting for a cmos camera should be gain to get max dynamic range, I think one should adjust the gain so that full well depth matches the max bit depth of the camera (e.g., 12 bit)? It is true that the camera may become non-linear before the pixel well is filled so it is necessary for one to measure the linearity of their camera to determine if the highest electron count should be less than 15000 (in this case) to avoid non-linearity. If that is found to be true, the gain setting should be slightly different from 3.66. It almost certainly will be?
I think the second setting should be software binning value to match image scale and seeing (fwhm). I think this is true for ccd cameras as well. It keeps from being undersampled or oversampled with the resulting increase in noise. This means some setting that will give "around" 2-3 pixels per fwhm. It does not need to be exact. In fact, since one can only bin in whole numbers, it would be 1x1 or 2x2 or 3x3, etc. One choice will be close enough. Since most cmos pixels are so small, software binning more than 1x1 is usually needed.
I think the third setting/adjustment (focus) you mentioned is related to both saturation and pixel/seeing match. That is, you want to get close to the nyquist value (2) when whole number binning doesn't get there itself. So, you may defocus to spread out the star profile over more pixels. This both reduces the chance of saturation because the photons/electrons are spread over more original pixels and it spreads the star profile over more pixels which may make you less undersampled.
I think the fourth setting (exposure) is obviously available to reduce the chance of saturation. For brighter stars, we would always (ccd or cmos) reduce the exposure so that fewer photons/electrons are collected. I think this is done after the above settings are selected. As you noted, as the exposure is reduced other sources of noise become more important/significant. IMO, the best way to reduce this noise is to increase the number of images taken and that can be stacked later. CMOS cameras with their electronic shutters can image at really short exposures. At some point less than 5 secs (or so), scintillation gets so significant that stacking is necessary.
IMHO, I think the important parameters should be set in the order noted above. They are all important to consider and to utilize under the full range of conditions we run into? I propose that one should not just try to vary all of the possible settings noted above in some random order. I think the settings should be made in the order listed.
IMHO, it is best to set/fix gain, then binning, then normal defocus amount (if necessary at all) to standard values for your system and then use exposure/stacking to achieve good photometry. At least most of the time. Nothing is ever perfect! ;-)
Comments/thoughts/disagreement are appreciated.
The ASI183 pixels are only 2.4 microns and that is well above the minimum 3 pixels per FWHM unless the images are taken with a camera lens that isn't a telephoto or the images are taken from the top of Mauna Kea where the seeing conditions are 0.5 arc-seconds. So why bin at 2x2 and defocus anyway?
I'm sorry that my approach has been somewhat heretical. I need to do more testing based on a traditional approach to imaging for photometry.
Basically, I have used a trial and error process with aim of finding the settings (gain, binning, exposure, degree of defocus) that yield the lowest possible standard deviation of a set (usually 10) of measured magnitudes. The settings arrived at will depend on the brightness of the star. So far, I have found no improvement in precision with binning 2x2 over unbinned images, but further testing can be done. Bear in mind, as discussed, with my CMOS camera "binning" just means averaging the counts across the binned pixels after readout has occurred. It is not physical binning.
The way I have been thinking, rightly or wrongly, is this. I thought that setting gain to get maximum dynamic range, up front, before looking at the magnitude range of your targets is not appropriate. There is no reason to do that with my system for, say, a 9th or 10th magnitude star. Max dynamic range for my camera is achieved at 5e-/ADU. For any one degree of focus/defocus an exposure 5x that at unity gain would be needed to reach the same ADUs. With my small aperture and fainter stars, I would be throwing away useful signal if I decreased gain.
However, after your comments I can go back and repeat the testing: set maximum gain, use (say) 9th mag targets, defocus minimally (trying to use the traditional approach with seeing, Nyquist number), try unbinned, binned 2x2 and binned 3x3 (if the formula permits - it may not). ZWO claims that with gain of 5e-/ADU full well is 20k (versus 4k at unity gain). Maybe that will allow photometry on more tightly focussed images. If so, it will be interesting to see if the actual precision is better with that approach than with my original one.
Just a comment on stacking. That's fine for tagets with long periods where you can take multiple exposures during one imaging session. Amost all of my targetst are short period variables (delta Scuti stars) with periods of 1 to a few hours, and I take time series. Therefore, stacking would be a pain. The alternative would be rolling averages, not so much of a pain, but that would drop the peaks and raise the troughs of the light curve. I prefer to avoid that.
Cloud has been a major problem (for astronomers) for the past couple of months. It's not going to clear substantially for a while yet. As soon as it does, I'll try the above, and post the results.
As far as I am concerned, you are following an experimental approach. I think it is the BEST way to do things. However, my hope is that we come up with a methodology that is both technically sound and practical so that others do NOT need to go through the same pain. Keep in mind that I do strongly feel that a little experimentation is also the best way to LEARN rather than just depending on what someone tells you!
The fact that the maximum dynamic range requires the highest gain (5e/adu) but yields the least sensitivity (it takes 5 photons/electrons, not one, to give one adu count) is the most difficult finding to deal with? It means you have to expose longer. So, should we use one gain extreme or the other or some intermediate compromise? I think I have made my choice and you have made another. Which is best? Note that for the ZWO ASI183 camera on BSM_NH2, gain settings are generally left at one value for all/most plans covering a range of target types and magnitudes. Just exposure is changed. It gets to be impractical / inefficient to modify other settings while remotely running the system.
Concerning stacking, note that it is only really required for bright stars that need short (<5 s) exposures. Even for your delta scuti stars that exposure gives you >> 100 images over the full period. If you use VPhot, stacking is quite easy. Alternatively, you could just aggregate groups of images (e.g., groups of 5) to average out the random noise. You do need to be careful to avoid averaging out real fluctuations. BTW, it is hoped that in the near future, that MaximDL will allow real-time stacking of images during the observing run so you are not forced to store a large number of short exposure, very large images (e.g., 50MB) on your hard drive.
I await your experiments with anticipation. This is great!