questions for submitting large amount of data

Affiliation
American Association of Variable Star Observers (AAVSO)
Sun, 11/18/2012 - 21:24

Dear list,

With a friend we are running a nova survey (called Dauban Survey) since April 2012. It is briefly described here: www.aspylib.com.

Since September 2012, we detected the variations of >500 known variable stars, and we have measured at least one data point for each one of them every week. I expect the number of variables will easily rise to a few 1000s in the next year. I hope we will have a good coverage of some Mira curves. Observation of known variables is not the primary goal but we would like if possible to submit the data to AAVSO.

At the moment I am going through the description of the "extended format" for submission, and try to see which format we could use. I have a few questions:

1) the system we use is a refractor 80/600 + CCD ST8300 + filter Astrodon exoplanet (cuts everything below 500nm).

We cannot use a standard photometric filter because we loose too many photons and we cannot spend too much time on a given field, in average we observe > 400 fields every night. The Astrodon filter is interesting because it cuts the blue part of the spectrum which is the most variable. The calibration is done with R magnitudes from USNO-B1 catalog (I know it's not the best one but that's the one we use for the astrometric reduction..).

My question is: what kind of filter should we indicate in the AAVSO extended format ? I guess CR is not correct because we have a filter. I see the possibility to indicate O = other filter but it does not seem to be recommended ?

2) for the magnitude calculation we use an ensemble photometry method. We have a fixed set of about 150-200 reference star for each field. We measure the magnitude constants (difference between USNO-B1 magnitude and instrumental magnitude = 2.5*log(flux)) for all these stars. We use the median of all the magnitude constants for our calibration. Using the median allows to remove the outliers and possibly a few variable stars. In principle the method should be very good. So far we did a few comparisons with other softwares and observers, and everything seems quite ok.

Here is my problem: for submission using the ensemble photometric method, I read that AAVSO requests a check star: http://www.aavso.org/aavso-extended-file-format

I don't know how can we find such check stars. If we pick up stars randomly, then I am sure, just by statistics we will pick up a few ones which vary. My impression is that adding a check star may be a source of error. Will the data be accepted without check star ?

Thanks for your help

Clear skies

Jerome Caron

Check star is required

Jerome,

Yes, a check star is required.  Without a check star, we have no measure of the reliability of your photometry.  While you can state the reliability broadly in your post, someone using your individual measurements from the database has no idea about the photometric quality without the check star.

You do not need to select check stars randomly -- if it is a variable with a comparison star sequence, you may simply use one of the existing sequence stars as a check.

But more fundamentally, regarding your photometry, clear filtered observations are ok for stars that are blue, but they are not good for red stars such as Miras.  Such data might be useful for measuring times of maximum when used by themselves, but will have limited use because they're very difficult to combine with other measures such as photometric V,Rc data or visual estimates.  Both the zero-point differences and color-dependencies will be difficult for someone using the data to overcome.

Matthew

Affiliation
American Association of Variable Star Observers (AAVSO)
Check star is for "insurance" as well!

I agree with Mathew.  Not only is a check star preferable where the referenced magnitude is shown next to your "determined" magnitude, but is also enables the system to reject bogus observations and saves embarrassment!  I had some of my observations of NSV 1436 rejected because my check star's magnitude was off by using the wrong reference star magnitude meant to measure against...my fault!

I re-did the reductions and used to correct reference star magnitude and all was fine!

One more thing for clarification.  I believe AASVO term "check star" is the 1st check star you assign.  This is done immediately after you pick a "reference" star, as in MaximDL.  This is usually the brightest star that doesn't saturate your CCD and assign it a standard photmetric magnitude.

James-FJQ   

Affiliation
American Association of Variable Star Observers (AAVSO)
Dear James and

Dear James and Matthew

Thanks for your answers. I think I definitely understand the idea of the check star. I have to think about it and see how I can implement this in our scripts. Probably we have to accumulate a bit of data to be sure that the check stars do not vary.

What about the filter ? Is that ok if I indicate CR for:  Clear (unfiltered), R-band comp star magnitudes ? In the end our curve of spectral response may still be the curve of some unfiltered CCD, even with the filter that cuts the blue < 500nm. Maybe I should also add a comment to describe the filter ?

I agree our setup is not well adapted to measure red stars.. But if we use a red filter, either we loose in magnitude or (if we take longer exposure) in coverage...

Clear skies

Jerome

blue cutoff filter

Hello Jerome,

Apologies for the belated reply -- we had a long holiday weekend here in the US.

As you say, a "CR" filter designation is probably the best match, since the magnitudes of the red stars won't really be influenced by the blue cutoff.  I would suggest three things if you are planning on submitting these to the AID: 1) make sure that you observe them for a long-enough period of time that we can try to understand the offsets between your data and the other data, 2) if these are rapid time-series, please submit nightly-averaged data for the red giant variables rather than rapid time-series, and 3) make a note of the filter in the comments for each observation. 

Unfiltered magnitudes for red stars can be very problematic when they appear in a light curve in isolation.  If we have several months or years worth of data from your system, we can try to understand them, but if you are submitting just a few observations per star, the red star data will be of limited use, and I would suggest withholding them.

Matthew

Affiliation
American Association of Variable Star Observers (AAVSO)
Dear Matthew

Thanks for

Dear Matthew

Thanks for your reply. I understand what you mean. With our setup we are getting one data point per night. To give an idea I have uploaded about 200 lightcurves on my website. It's about 2 months of observation. I think there will be a gap of a few months due to winter, and in Spring we will continue for a longer period (maybe 6-8 months) over the same area.

http://aspylib.com/survey/lightcurves_2012-11-27/

The work is not finished, I need to add a checkstar, compute the mag limit of the sky background (to filter out values such as mag=20!) and compute mag uncertainty. But it gives an idea of our sampling. Also our algorithm only detect the stars that increase in flux (because we look for novae and outbursts) so we will pick up other stars in 6 months, the ones that decrease at the moment.

Basically, I would like to know which ones of these stars are interesting for AAVSO ?

Most of the stars we found are actually red variables, Mira and SR.  Some of them do not have any known period so the data should be interesting. But what about the well-known stars, should we also submit them ? For the short periodic stars, they are clearly undersampled. I hope that in the long run we will be able to "fold" the lightcurve and get a correct phase plot (e.g. for the RRLyrae and eclipsing stars).

Clear skies

Jerome

Affiliation
None
survey

Hi Jerome,

Congratulations on starting a new survey.  My understanding is that this is a novae survey, so I am a little surprised that you say adding a photometric filter will decrease the light too much.  You are already getting decent light curves at mag 13; adding say an Rc filter would only decrease your throughput by a magnitude or so, still sufficient to detect any nova expected in the northern Milky Way (much less extinction than, say, Sgr, and less distance to reach the outer edge of the galaxy).

I'm glad that you are planning on adding photometric uncertainty.  However, even with that, there are a couple of things to watch out for.  Taking V15.png for example, which is the light curve for R Vul, your measures cover 9.4-8.3mag during an interval when the Vmag was 13 to 11.  This would imply to me that you have a scale error in your photometry; you are including nearby companions that are limiting the faintness of the variable; or it has much lower amplitude in your bandpass than at V.  Probably the latter is the correct assumption, as your filter will act primarily like Ic with red miras and not even CR.

If you want to maximize the scientific return, you might consider switching to a photometric filter.  Then your observations can be used for many different projects beyond the novae searching.  For short period variables, sparse sampling of the light curve is fine as long as you already know the period - it is period searching where the sparseness causes aliasing.  We have many good light curves of RR Lyr and Cepheid variables taken with BSM-NM, where observations occur once per night.

You talk about perhaps a thousand variables with perhaps hundreds of data points per star, so a data volume in the few hundred thousand observations range.  There are several ways to consider submitting the results:  (1) create a database of the observations, and have us link to them or mirror them at HQ, and not specifically submit them to the database; (2) submit only the non-red objects, like binaries or short-period pulsators, where your exoplanet filter will result in measures much closer to CR; (3) submit everything and let it be caveat emptor.  We should consider the best approach.

Some comments about your photometry.  What magnitudes are you using for comparison stars; are they the Cousins R values or Johnson V?  I assume the red points for V15 are for saturated measures; those are best classified as "brighter than".  I would delete these before submission as they have far less value.  I am somewhat concerned that there is a "jump" in the magnitudes rather than a smooth transition to saturation - perhaps you changed exposure times or something?

Thanks for considering the submission of the photometry to the AAVSO.  We'll figure out some way to make it work!

Arne

Affiliation
American Association of Variable Star Observers (AAVSO)
Dear Arne

Thanks for your

Dear Arne

Thanks for your message. I try to answer the questions. Yes, our main purpose is to search for novae. For the calibration we use R magnitudes from USNO-B1, with a large number of comparison stars (>150). For the V15.png (R Vul) the red dots are saturated values and the magnitude that is plotted is meaningless, it is just set to some artificial value.

About making filtered photometry: I understand your points. We are discussing about it at the moment. I think we would need the R filter for the novae due to the Halpha line. My fear is that it would degrade the photometry, for the red stars we will maybe loose 2 or 3 magnitudes. We are also following CV stars, and we may loose many of them because they are close to the limit. I agree in the long run we have to find a solution.

About the best way to submit the data, which solution would be preferred by AAVSO?

(1) create a database of the observations, and have us link to them or mirror them at HQ, and not specifically submit them to the database; (2) submit only the non-red objects, like binaries or short-period pulsators, where your exoplanet filter will result in measures much closer to CR; (3) submit everything and let it be caveat emptor.

Another question: we have detected many variable stars which are very close to the position of a NVSV, ASAS or ROTSE variable. I understand we have found the same variable star, but it's just that the NSVS/ASAS/ROTSE astrometry is inaccurate. In all cases, the NSVS/ASAS/ROTSE position in VSX corresponds more or less to the center of a group of stars, and in our images we are able to identify exactly which star is varying. We have several such cases, maybe (on top of my head) 20 to 40. I am sure this information is interesting. What should we do with it ?

Thanks for your help

Clear skies

Jerome 

One size doesn't fit all

Hello Jerome,

This conversation brings up a number of issues, most notably that the number of surveys are proliferating, and we need to find good ways to make data from these easily obtainable.  It also highlights the fact that surveys cannot do all things simultaneously.  Systems that are optimized for faint CV monitoring are not ideal for long-term photometry of red variables (and vice versa).

Regarding the filter question, there really isn't a universal bandpass that you can use to submit data, because there are large variations in spectral properties redward of the cutoff of your filter.  Your photometry of a G star will have a substantially different response than that of an M star.  As Arne mentioned, you detected R Vul at a magnitude of around 9.0 when the V and visual observers had it around 13 to 14.  That's likely an I magnitude rather than an R, so even CR isn't appropriate.  But if you were doing photometry of a eclipsing binary with only modest B-V colors, your system would be fine.

From Arne's list of options, option (1) might be most appropriate if you are able to serve the data easily.  Otherwise, I would suggest only submitting a subset of your stars to the AID, mainly those that have V-I colors blueward of about +1.0 to 1.5.  That will substantially restrict the number of stars that you would submit to the AID, but again, it would be hard to understand your observations when plotted against data with better-understood photometric properties.

Regarding your second question, if the NSVS or ASAS objects do not have better localizations from other catalogs, then by all means you should submit the revised positions to VSX.  That is especially critical for cases where you have several stars within 30" or so that could be candidates for the variable.

Thank you for discussing these data so thoroughly before moving forward!  I think this is valuable work, regardless of how the data are made public and accessible.

Matthew

Affiliation
Royal Astronomical Society of New Zealand, Variable Star Section (RASNZ-VSS)
Questions for submitting large amounts of data

Greetings, Arne and all,

I was looking at one post a few days ago where someone was encouraging CCD V measures of Mira stars which seemed to be rather unnecessary - the visual observers are doing a good job there and with the accumulated knowledge of the last century and more it takes very few measures to keep abreast of how these stars are performing - with the exception of the half a dozen or so which are behaving spectacularly.

This project seems to be in its infancy and will probably change. Since novae are blue objects at birth the idea of a long pass filter seemed strange. If this project can produce good data on some of the really underobserved variables like SR and LPVs it would be useful. But I'm reminded of Sergei  Gaposchkin and Cecilia Payne's measures of LPVs a long time ago in something they called 'I' which was a non-standard band and was hard to use in determing epochs of maximum. So salecting a standard filter seems essential if they're to be incorporated in the main database. The idea of a linked database seems sensible and could even be extended to some of the other measures like unfiltered photometry of CVs.

We should be concerned that the large influx of automated and semi-automated CCD measures seems to be affecting the numbers of visual observers. Many of these projects start off with great intentions but then disappear for various reasons. But before they do they often have unexpected consequences. I think the offer is premature and needs attention to the points raised by you.but I really like the idea of setting up a linked database for all of these high volume projects.

Regards,

Stan

WWSG, which I'll be using shortly!