Differences

This shows you the differences between two versions of the page.

--- start:report_ais [2018/11/15 17:09] – hanany
+++ start:report_ais [2018/12/05 14:31] (current) – [Michael Strauss] kyoung
@@ Line 25: / Line 25: @@
   * Too much technical information. Shorten technical section.
   * Be clearer about the need for space relative to ground/balloon
+==== Kathy Romer ====
+Hi Shaul
+Yes, feel free to quote me. It was off the cuff, but sometimes those are the most true things we say! (well unless you live in the White House!)
+Here are some things that I think a) should be changed b) could be changed. I expect most in the latter category can't be changed because time is too short, plus I expect all are a matter of taste/style and so would need to be considered by the PICO team before implemented. In the unfortunate (and I hope unlikely) event that it isn't approved this time, then you could consider them for next time.
+General comment: assuming PICO is affordable (i.e. costs less than LSST) then this is a no brainer, the science is varied, fundamental and compelling. The only issue I see is that people who do similar science (be it using CMB or not) will feel elbowed out and if they are on the panel then they might not be able to get over that self-preservation bias. So what would be great is if the proposal included some suggestions about how to build a large community around PICO in the years up to launch.
+FYI I haven't read past page 34.
+Best of luck with the bid. Thanks for asking me to review it, I learned a lot.
+Kathy
+a)
+You didn't "big up" the papers already submitted about PICO. I am guessing they are in the list of 186 references, but could have more prominence. I see 3 on ADS (sadly a bit lost in the noise due to some bubble chamber with the same name).
+Executive summary - I though it a bit odd that you didn't mention EUCLID, SO or SKA, given that WFIRST and LSST are mentioned.
+Executive summary - I flagged up the protoclusters and noted "how - no ICM?" and only got an answer many sections on (via star formation). So you could add a note about that.
+Table 1 - I couldn't read it at all: I printed 2 pages per side, so it came out tiny. Even on the .pdf version it looks too small. To read it I'd need to zoom in. So I didn't. You could try rotating it or printing it across two pages. [BTW I really liked the graphic on the cover page - that was very clear and persuasive]
+Figure 1 - You probably said this somewhere, but I missed it: EE spectrum is not the same as TT spectrum: most people are very familiar with the latter by now, so would be puzzled by figure 1.
+Section 2.1 - para 2 - should it be "Galactic science" or "Milky Way science" rather than "galactic science"? Or maybe you did mean galaxies in general?
+Section 2.1 - para 4 - "Its measurements of the spectral index" -> "PICO's measurements of the spectral index"
+Section 2.1 - para 5 etc. - you refer to S05 but didn't refer to S01-04 in preceding paragraphs. As someone who couldn't read the table, it wasn't clear to me at first what SO* meant (unfortunate overlap with SO, see below)
+Figure 3 - you have an "SO" annotation, but this isn't explained in the caption.
+Section 2.2.1 - Dark Matter bullet - you say "interactions between dark matter and protons" and that puzzled me. What type of interaction, given that DM is usually thought of being WI at best?
+Section 2.2.2 - Cosmic Birefringence bullet - when I read this I worried that this effect would be degenerate with CMB lensing and/or primordial B modes. This no doubt shows my ignorance of the field, but maybe some of the official reviewers might be similarly ignorant.
+Section 2.2.2 - Formation of First Luminous Sources part - I highlighted "higher resolution than PICO": that felt like a bit of an own goal (by suggesting that PICO isn't as good as other experiments). So maybe that could be phrased differently to sound less of a negative. Similarly the bit "while PICO alone may not enable high SNR...". [Although the ending comment about complementarity was strong, so definitely keep that in.]
+Section 2.2.2 - Formation of First Luminous Sources part - I don't think "Stage-III CMB" was defined. I know what you mean, but maybe not everyone would.
+Section 2.2.2 - Structure Formation via Gravitational Lensing part - "would represents" -> 'would represent"
+-> Done
+Section 2.2.2 - Structure Formation via Gravitational Lensing part- It wasn't clear to me how you could "effectively cancel cosmic variance". [BTW I found this section compelling, it is one of the few where I have useful background!]
+-> Marcel: Edited slightly: "For instance, one can use correlations between large scale structure tracers with different clustering bias factors and measure the relative difference between their clustering power spectra to effectively cancel cosmic variance~\citep{2009PhRvL.102b1302S,2018PhRvD..97l3540S}; this can constrain..."
+Figure 11 - I don't understand what you are trying to show with the full sky and with the zoom ins (and why two zoom ins of the same region)?
+Section 2.2.3 - Dust Physics part - you say "levels relevant for B-mode science" but don't refer back to the section where this was explained.
+Section 2.3 - I think there needs to be something at the start to segue from 2.2 to 2.3. A couple of sentences as to why the Legacy stuff is different to what has already been talked about. Are you expecting other people to do this science?
+Section 2.4 - I thought it was very noticeable that ground based (rather than balloon) based CMB experiments were not discussed. There are several kind things said about balloons, but nothing about SPT/ACT/SO. That seems risky given who'll likely be on the panel.
+Section 2.6.4 - for some reason this section worried me. It seemed like this issue might be a show stopper. I'm guessing it isn't, so maybe you could soften the language a bit.
+Section 2.6.4 - "future space mission as PICO" -> "future space mission such as PICO"
+b)
+Why no table of contents?
+Executive summary - not being an expert, I underlined "cosmic birefringence" and added "?". So for specialist terms like that you could add a link to the relevant section.
+Executive summary - "future cluster redshift surveys" is a bit vague and probably inaccurate. These days we measure cluster redshifts from photometric data. LSST is not specifically a cluster redshift survey, but it will measure cluster redshifts to z~2 (i.e as long as there is a red sequence - it won't do so well with star forming cluster galaxies).
+Sectioning - general comment: this is a very long proposal and the hierarchy does get a bit confusing. Section 2 is "Science" but I don't think 2.5 - 2.7 fit under that heading - maybe better as Section 2 "Science background: Cosmology and Physics", Section 3 "Science background: Other", Section 4 "Technical Requirements" (with current 2.5-2.7 in Section 4).
+Section 2.1 - this section has a lot of detail in it, but not everything is explained (e.g. the meaning of "quantum gravity" or how "specific models of inflation" would be tested) so I think it would benefit from referencing ahead to the sections where more information is given. And, on the flip side, maybe this section could have been shorter and at a more general ("for dummys") level because a lot of what is in it is repeated later on.
+Glossary - a glossary might be useful if you have room within the page limits.
+Section 2.2.2 - Formation of First Luminous Sources part - I got lost because it started talking about large scales and then talked about small scales. Probably me not paying close enough attention, but maybe the official reviewers might do the same.
+Structure of sub-sub-sections: this is definitely a style/taste thing, and for a future bid (which hopefully won't be needed) but one thought I had was that you could structure the sub-sub-sections (sometimes sub-sub-sub-sections) to bring out the "elephant in room" e.g. why do we have fluctuations in the CMB, the "state of the art and why the elephant remains hidden", e.g. Planck, SPT, balloons etc., "how PICO will unveil the elephant" (either on its own or with synergy with LSST, balloons etc.).
+Section 2.2.2 - Galaxy Formation via SZ...  part - the ordering of the sections doesn't quite work, since you define kSZ and tSZ here, but have referred to them already in earlier sections. May not be a big deal since most people will know what they are anyway.
+Section 2.2.2 - Galaxy Clusters part - similar point to above: you've already talked about using clusters to do cosmology and now you are telling us about the catalogue.
+Section 2.2.3 - Formation of Stars within... part - I wasn't sure if it was 1000's of independent measurements in time or spatially. I figured out that it was spatially through the context. related: why do you need so many measurements?
+Table 2 - I like the way this is broken down with "Current knowledge" summaries. But it only talks about Planck. What about ACT, SPT, SO, balloons etc.?
+Section 2.3.1 - page 22 - I noted "why do we care about lensed high-z galaxies" in the margin. So maybe you need to stress the motivation for this section more?
+Section 2.6.2 - this section fizzled out. I wasn't sure if you'd done the predictions or not.
+Section 2.7 - I found it a bit odd that the requirements are described here, when they have been alluded to throughout the preceding sections. But its not a big deal.
+==== Michael Strauss ====
+Dear Shaul,
+This is an exciting project indeed, and the writeup is very good!  I attach a PDF scan of it with my hand-written comments.  Most are minor typo corrections and suggestions on wording.  Just a few general comments:
+  *  As you state fairly late in the text, PICO is the obvious successor to the legacy of COBE, WMAP and Planck.  I think this basic theme should be stated explicitly from the beginning of the report.
+  * On a closely related note, the report makes multiple statements from the beginning about how much better PICO will do than Planck, by all sorts of metrics.  But the reader doesn’t know what PICO actually *is* until quite late in the report: a 1.5m space-based telescope with a 5-year lifetime to do an all-sky survey, and 22 logarithmically spaced bands, designed to minimize systematics, especially for measurement of polarization.  It would be good to have a clear statement of this from the beginning, to put all the claims about what it will *do* into the context of what it *is*.
+  *  Given that the current state of the art is Planck, it would also be good to clearly state what distinguishes PICO from Planck.  After all, they are both are ~1.5m telescopes carrying out full-sky surveys in multiple bands.  I *think* the answer is:
+   - Far more detectors on the focal plane (with better noise characteristics).
+   - More than twice as many filters (although the range of frequencies is quite similar).
+   - Far superior control of systematics in the design, especially for polarization.
+   - Considerably better angular resolution.
+This should be stated from the outset.  In this regard, the requirements for the mission are somewhat spread out through the document; first appearing in Table 1, but then only being explained right at the end (Section 2.7), which is slightly confusing.
+OK, more in the attached PDF.  You’ll note I have only a handful of comments on the technical/instrumental part of the report.  Do let me know if any of this is unclear.
+This is a *really* exciting project; I do hope it goes ahead!
+Cheers,  Michael
+{{:start:strauss_comments.pdf|Strauss Comments}}  Karl and Shaul currently going through these handwritten comments (Nov. 28-??)
+==== Michael Shull ====
+{{:start:shull_comments.pdf|Shull Comments}}
+==== Douglas Scott ====
+   * {{:start:douglasscott.pdf|Douglas' first round}}
+In any case, feel free to post my detailed comments.  General issues include:
+   * it would be better to include pointers to SO1 and SO2 first, since otherwise this labelling in the text is confusing
+   * there are places where the text should say "PICO will" rather than "PICO would"
+   * I hope that later (which I haven't gotten to yet) there's a discussion of comparison with ground-based efforts (in particular CMB-S4) - plus is it clear whether to assume Litebird will happen or what?
+More to come.
+ Douglas
+Here's another set of pages (up to p.19 now).
+   * {{:start:pico_nextdozenpages.pdf|Douglas' second round}}
+More general comments:
+   * I think I agree with Martin that the structure doesn't seem optimal because it's driven by methods rather than science questions.  But rearranging things at this point would be quite painful to do.
+   * Presumably we can claim that detailed modelling of the dust polarization stuff will ultimately help with removing Galactic polarization in ordet to get at primordial signals?  That's probably not trivial to show woth simulations, because the simulations aren't complicated enough - but the only way we'll discover that dust behaviour is more complicated than we thought it was is by studying the dust signals.
+   * It's odd to motivate birefringence by saying that it might have something to do with an explanation for acceleration.  Well maybe, but I don't think that's the usual story.  Mostly people are just saying there are possiblities for this kind of coupling, so let's keep an open mind and see if you can constrain the signature.  As well as the Polarbear paper, the relevant Planck paper (Intermediate XLIX) should be cited.
+   * Is f_NL=1 really well motivated by theory?  Maybe this is the strongest statement that can be made, just because this is the value that's doable for PICO?  But can someone spin this a bit more convincingly?
+Here's the last set of detailed comments on the science part (pages 20-35).
+   * {{:start:pico_lastsciencepages.pdf|Douglas' third round}}
+General comments on this part:
+   * It can't really be true (as stated in Table 2, that polarization has only been measured for one "dusty galaxy" - unless we have some restricted meaning (above some z and L?)
+   * The text says that Planck's resolution was too poor to detect individual proto-clusters - but I've written many papers on these, so that's just not true!  The easiest way to fix is is to say "too poor to resolve individual proto-clusters" - certainly Planck's 5' beamsize wasn't optimal for studying such sources, and PICO will be much better.
+   * Is it worth saying "SPHEREx if selected" - isn't that just a red flag for any SPHEREx fans?  The text doesn't use such language for other future projects.
+   * In the version I'm using, the text that straddles pages 23 and 24 is a HUGE red flag!  It's about the relative costs of ground-based versus space-based, and I don't think the wording as currently written would be agreed by everyone!  Please be VERY careful to reword this part - maybe there's a different way of spinning this whole thing to sound more positive about space without getting into the dangerous territory of making claims that not everyone agrees with?  We're surely not claiming that space is now cheap?!
+   * And why is there no explicit mention of CMB-S4 at this point?
+   * In fact CMB-S4 is introduced (I might even say buried) in the middle of Section 2.5.2, which seems very odd.  I think this comparison (and obviously the thing to do is stress the complementarity rather than the competition) needs to be called out more explicitly in a subserction of its own.
+   * Around the bottom of page 28 and top of page 29, I'd add some text (suggestions included on the pdf) about the possibility that the foregrounds will be more complicated than anticipated - this is a big issue in people's minds, and doing things properly with PICO is surely a major selling point.
+   * It seems odd that the discussion of systematics is a subsection of Section 2, rather than being a Section on its own.
+   * I became less and less of a fan of "IGW" as I read this.  Why introduce this new acronym?  Not everyone will know what it means, PLUS it assumes that the signal will be from inflation, which precludes the possibility that we'll measure something and find out that there's a non-inflationary eplanation (howwver unlikely that may seem, it seems unwise not to at least SOUND open-minded!).  Why not just say "primordial GW" or just "GW"?
+   * Are you required to write out all the authors in the reference list (it looks sloppy to me, like no effort was made to reduce the list to the first N).
+   * Lastly - I see that the technical parts of the report are in a completely different format.  I think that gives a very bad message, namely that the science and technology parts aren't fully integrated.  Maybe it's a lot of work, but it would be much better to format all of it the same way (maybe the intention is to do this before submission?)
+==== Masashi Hazumi ====
+Dear Shaul,
+My major comments follow:
+  * Assuming that PICO comes after LiteBIRD and CMB-S4, the improvement on r is not dramatic, factor 3 or so (sigma(r=0)~0.0005 both from CMB-S4 and LiteBIRD with external data). You may need to choose some other killer applications to sell PICO. Of course it could be the case that neither of them is accepted, but it is not a good strategy to ignore these proposals.
+  * The science goal of sigma(r=0) < 10^-4 is experimentally very ambitious. Since there is no quantitative result on foreground cleaning and systematic error estimations, I cannot judge if this goal is achievable at the moment.
+  * Sec.2.6.2 seems to say sigma(r=0) < 10^-4 is NOT achievable with using TB and EB. Is that your main message?
+  * Bandpass mismatch and its stability during observations is a large concern. Although Table 4 seems to say it is written in Sec. 2.6.2, no description is found.
+  * There is no description about interplays between foreground cleaning and instrumental systematic errors. To achieve sigma(r=0) < 10^-4, they could be the killer. So I suggest that you include them in your risk list.
+  * Have you considered effects of multiple-reflections and ghosting in your far sidelobe estimations? 80dB is very challenging.
+  * Lowest frequencies are on edge wafers. Have you checked that beams at the edge pixels are still OK for the lowest frequency?
+  * What is your plan to mitigate 1/f noise? What are your requirements? Detector 1/f is not an issue. 1/f is a system issue, coupled to energy deposits by cosmic ray hits, scan-synchronous instability, cooling system instability etc. It is hard to control. I suggest you add some more descriptions about 1/f noise requirements and how to deal with system issues related to 1/f noise.
+  * How do you deal with gain variation within 10 hours?
+  * There is no convincing argument that >450GHz is required for foreground cleaning. Foreground experts say that under the current foreground models considered, better sensitivity below 450GHz is equivalent to having >450GHz. Frequencies closer to CMB bands are actually better if you consider something unknown, because at higher frequencies you need to worry more about new components such as warm dusts. Somewhat related to that, COMMANDER results used to show the necessity of higher frequencies, but new COMMANDER-2 introduced this summer does not anymore. So if you happen to use COMMANDER, results should be updated.
+  * You quoted S4 results, but simulation results from S4 and SO are optimistic compared with the one for LiteBIRD (by Josquin and Radek arXiv:1811.00479) because they do not take into account possible spatial variation of the synchrotron curvature. The SO science paper (arXiv:1808.07445. Sec3.5 limitations of current forecasts) also mentions it as a limitation. Since the description of foreground cleaning in this document is short, I cannot judge if this is also the case in your study. Since the foreground cleaning is a key to the success, I suggest you describe more details.
+  * In general, space agencies appreciate if you give a thorough list of things that need to be addressed, such as the interplay I mentioned above, rather than ignoring them and giving an impression that you do not recognize them.
+Minor comments
+  *  2 < ell, do you mean no sensitivity is required for ell=2?
+  * How do you avoid CO lines?
+==== Paolo deBernardis ====
+- Optical system: From fig.3.1 left and fig.3.2 it seems that there is a direct path from radiation diffracted from the lowest edge of the primary mirror to the focal plane (with a large incidence angle, but still after a single diffraction). Is there any way to baffle it, may be moving up a bit the focal plane ?
+- Polarization modulation: There is no polarization modulator, and polarization modulation is achieved through an optimized sky scanning. According to the simulations mentioned in the proposal, accurate polarimetry can be achieved nevertheless.  However, the reviewer would expected to see more details on the simulations, to evaluate how realistic and reliable they are. This is important since LiteBIRD, sharing some of the main goals of PICO, has implemented a very different strategy, stating that the presence of a polarization modulator is mandatory.  The main problems to be considered more quantitatively are, in my view:
+a) efficiency of the mitigation of the effects of 1/f noise (the discussion at the end of paragraph 4.1 is really short and not quantitative)
+b) effect of beam ellipticity, coupling anisotropy to polarization
+It would be good to add a column in Table 4 with upper limits to the bias/error affecting the determination of r, as estimated from end to end simulations, for each of the systematic effects listed. I understand that more detailed analysis will be carried out in phase-A, but the reviewers should be reassured that the evaluated risks are under control.
+- In the caption of fig.5.2 the general layout is the Planck one, but the focal plane of Planck was at 100 mK while 250 mK is indicated.
+I'll continue to go through, but wanted to transfer these.
 ==== Michael Strauss ====
@@ Line 196: / Line 412: @@
 ) "...More than 60 people participated in-person in two community workshops (November 2017 and May 2018)." --> "...More than 60 people participated in-person in two community workshops (November 2017 and May 2018). As such, this study represents broad contributions and technical experience from the CMB theory, analysis and instrumentation communities." or similar.

User Tools

Differences

Page Tools