Skip to content
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
310 changes: 310 additions & 0 deletions doc/examples/POLREF/Ni58_d.dat

Large diffs are not rendered by default.

69 changes: 69 additions & 0 deletions doc/examples/POLREF/Ni58_model.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
from refl1d.names import *
from refl1d.probe.data_loaders.polref_legacy_data_loader import load_probe_polref

dQoQ = 0.01
theta = 0.25
probe = load_probe_polref(
filename="Ni58",
angle=0.25,
dQoQ=0.01,
name="Ni58",
pol_mode="pnr",
intensity=1.0,
background=1e-7,
back_reflectivity=False,
)

probe.pp.intensity.range(1e-1, 10)
probe.pp.background.range(1e-9, 1e-3)
probe.pp.sample_broadening.range(-(dQoQ * theta), 0.03)

# Set materials/SLDs
Si = Material(formula="Si")
Ni = Material(formula="Ni[58]")


Si_sub = Slab(material=Si, thickness=0, interface=5)
Ni_layer = Slab(material=Ni, thickness=1200, interface=5)

# Sample construction/Stack

sample = (
Si_sub
| Ni_layer(magnetism=Magnetism(rhoM=2.0, interface_above=5, interface_below=5, name="Ni Layer Sample 1"))
| air
)


# Fit params

Ni.density.pmp(-50, 0)

sample[Ni].magnetism.rhoM.range(0, 5)
sample[Ni].magnetism.dead_above.range(0, 100)
sample[Ni].magnetism.dead_below.range(0, 100)
sample[Ni].magnetism.interface_above.range(0, 50)
sample[Ni].magnetism.interface_below.range(0, 50)

sample[Ni].magnetism.rhoM.tags = ["magnetism", "sample"]
sample[Ni].magnetism.dead_above.tags = ["magnetism", "sample"]
sample[Ni].magnetism.dead_below.tags = ["magnetism", "sample"]
sample[Ni].magnetism.interface_above.tags = ["magnetism", "sample"]
sample[Ni].magnetism.interface_below.tags = ["magnetism", "sample"]


Ni_layer.thickness.range(0, 1500)
Ni_layer.thickness.tags = ["structure", "sample"]

Ni_layer.interface.range(0, 50)
Ni_layer.interface.tags = ["structure", "sample"]

Si_sub.interface.range(0, 50)
Si_sub.interface.tags = ["structure", "sample"]

zed = 2
step = False

experiment = Experiment(probe=probe, sample=sample, dz=zed, step_interfaces=step, auto_tag=True)

problem = FitProblem(experiment)
309 changes: 309 additions & 0 deletions doc/examples/POLREF/Ni58_u.dat

Large diffs are not rendered by default.

148 changes: 148 additions & 0 deletions refl1d/probe/data_loaders/polref_legacy_data_loader.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
from refl1d.names import PolarizedNeutronProbe, NeutronProbe
import numpy as np
import os
from refl1d.probe.resolution import QT2L
from pathlib import Path


# TODO: See if we can wrap np.geomspace to give the same behaviour as below?
# Currently, np.geomspace does not allow for a step size to be defined,
# only the number of points.
# This is not ideal for TOF data where we want to define a step size in dQ/Q.
def logstep(start, stop, step, base=10.0):
"""
Creates a log spaced 1d array by defining a step size and a base
In the form of dQ/Q - i.e. dQ\\Q*Qpoint
"""

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use: np.exp(np.arange(log(start), log(stop)+log(1+step), log(1+step)))

The result is independent of base.

logrange = [start]
point = start
while point < stop:
point = point + base ** (np.log10(step * point) / np.log10(base))

logrange.append(point)

return np.array(logrange)


def TOF_loader(T=0.25, dQoQ=0.02, Q_sim_range=(0.005, 0.2), filename=None, skiprows=1, **kw):
"""

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest load_TOF (filename, T, dQoQ=0.02, ...).

load_TOF because function names are usually verbs (classes are usually nouns).

filename first because that corresponds to the order in the other loaders

T without default since you don't get theta offset and sample broadening otherwise. I'm assuming there is no good default value for theta.

dQoQ can default to 0.02 if that's what usually comes out of the reduction software. Specify in the docs that it is FWHM. If your usual Q spacing is set to dQoQ, then you can infer dQoQ from the Q spacing in the data.

Overloading the interface with filename=None to ask for a simulation is a little awkward, but I suppose it is okay.

Given that your wavelength range is relatively stable, I would expect the simulation to ask for an optional L_range=(low, high) that you can override if necessary instead of asking for Q_sim_range. [I'm assuming you are using theta control the instrument and not Qmin for each segment.]

If the L range is standard, then I guess you can infer theta from Q. That is, either T or L_range could be used as inputs. If reduction has a default L range in common use, then make it the default so you don't need to specify T or L_range when loading the file.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@acaruana2009 Is there a significant existing set of models that use the existing signature? If so, I would argue that maybe keeping the less-than-ideal current signature makes sense, especially since this loader function is being added largely to support past and current models (with future models using the ORSO loader instead)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the polref loader isn't in refl1d yet, existing models must be defining the reader in the script. None of these will find the new interface, so you are free to change it without breaking backward compatibility.


We will want a data loader as part of our model builder interface. This will probably look at file extension to choose the appropriate reader, or maybe it will cycle through all registered formats until it finds one that works. Without T and dQoQ, the polref reader can only return a Q probe without theta offset and sample broadening parameters.

We could provide a reader hook in webview which asks for this information but this will be hard to maintain. Instead have a separate tool that converts a directory from legacy format to ORSO and load the converted file into the model.

Loads and creates NeutronProbe objects for TOF stitched datasets
I.e. from multiple angles. In the case of ISIS NR instruments we
typically have a constant dq/q resolution which the data is binned to at
the end of the reduction.

*T* incident theta for the lowest angle
*dQoQ* dq/q resolution data has been binned to
*filename* filename of the data set to be loaded
*kw* keyword arguments (kwargs) to be passed to NeutronProbe()
"""

# np.loadtext is currently set for simple 3 column POLREF data
# if you aim to use this loader for other data, talk to your local contact
# to understand the data format, and how best to load it.

if filename is not None:
data = np.loadtxt(filename, skiprows=skiprows).T
if dQoQ is None:
Q, R, dR, dQo = data
else:
Q, R, dR = data
dQo = Q * dQoQ
data_in = (R, dR)
else:
Q = logstep(Q_sim_range[0], Q_sim_range[1], dQoQ, base=dQoQ)
data_in = None

L = QT2L(Q, T)
# Converting the dq/q resolution into a dq value for each Q point
# dQ = FWHM2sigma(dQo)
# Since we take dL/L = 0, dQ/Q = dT/T, so dT = T * dQoQ
dT = T * dQoQ
# print(f"dT = {dT}")

probe_out = NeutronProbe(
T=T,
dT=dT,
L=L,
dL=0,
data=data_in,
# For standard TOF measurements resolution is assumed to be normal (gaussian)
# For measurements with many wavelengths and many angles (say cw measurements)
# then a uniform resolution can be used instead.
resolution="normal",
**kw,
)

return probe_out


def load_probe_polref(filename, angle, dQoQ, name=None, path=None, pol_mode=None, field=None, **kw):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like load_TOF, you may not need angle or dQoQ.

"""
creates one probe (Neutron, Polarized - PA or PNR) from one measurement - could be one angle or stitched dataset.
If polarized, sets some default values and links instrumental parameters for each cross-section together
"""

if name is None:
name = filename
if path is None:
path = os.getcwd()

filepath = Path(path) / filename

if (pol_mode != "pnr") and (pol_mode != "pa"):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could guess pol_mode by looking in the directory. One less parameter for the user.

if Path(f"{filepath}.dat").exists():
    pol_mode = None
elif any(Path(f"{filepath}_{pol}.dat").exists() for pol in ("u", "d"):
    pol_mode = "pnr"
elif any(Path(f"{filepath}_{pol}.dat").exists() for pol in ("uu", "du", "ud", "dd"):
    pol_mode = "pa"
else:
    error

probe = TOF_loader(T=angle, dQoQ=dQoQ, filename=f"{filepath}.dat", name=name, **kw)

probe.intensity.name = f"intensity {name}"
probe.background.name = f"background {name}"
probe.sample_broadening.name = f"sample_broadening {name}"
probe.theta_offset.name = f"theta_offset {name}"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are the default names given within the Probe constructor, so no need to set them in the usual case. If you are passing them in as independent parameters, or parameters from another probe, you don't want to rename them here.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See the latest commit. I have set the name in the correct place - this was probably left over from when I first wrote it years ago and wasn't aware of the name argument.


probe.intensity.tags = ["inst", "nuisance"]
probe.background.tags = ["inst", "nuisance"]
probe.sample_broadening.tags = ["inst", "nuisance"]
probe.theta_offset.tags = ["inst", "nuisance"]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These tags should be set in the Probe class.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what does it mean, to have them set in the Probe class? Do you mean these should be the default tags for every Probe instance?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These tags are not specific to polref data.

If we are going to set them in the loader, wouldn't it be best to do it generically so that they are consistent across loaders?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does that mean yes, you want to set them in the Probe init as the default tags?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we can add tags=("inst", "nuisance") to all the Parameter.default initializers in probe.py. Here are the ones for Probe:

self.intensity = Parameter.default(intensity, name="intensity" + qualifier)
self.background = Parameter.default(background, name="background" + qualifier)
self.back_absorption = Parameter.default(back_absorption, name="back_absorption" + qualifier, limits=(0.0, 1.0))
self.theta_offset = Parameter.default(theta_offset, name="theta_offset" + qualifier)
self.sample_broadening = Parameter.default(sample_broadening, name="sample_broadening" + qualifier)
self.back_reflectivity = back_reflectivity

I'm suggesting a tuple rather than a list because the input should be frozen. If code later tries to append to the list I want it to through an error rather than modify the default for the next call to Probe.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently if auto_tag (which defaults to False) is set in the refl1d.experiment.Experiment class, it adds the the tag "probe" to all parameters of the Probe object.

A fix is proposed in #387

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With #387 you can delete the above


else:
if pol_mode == "pa":
files = dict(
data_mm=f"{filepath}_dd.dat",
data_mp=f"{filepath}_du.dat",
data_pm=f"{filepath}_ud.dat",
data_pp=f"{filepath}_uu.dat",
)
else:
files = dict(data_mm=f"{filepath}_d.dat", data_mp=None, data_pm=None, data_pp=f"{filepath}_u.dat")

cross_sections = []
for data in files.values():
if data is None:
cross_sections.append(None)
else:
cross_sections.append(TOF_loader(T=angle, dQoQ=dQoQ, filename=data, name=name, **kw))
if field is None:
field = 0.0

probe = PolarizedNeutronProbe(cross_sections, Aguide=270, H=field, name=name)

for xs in (probe.mm, probe.mp, probe.pm, probe.pp):
if xs is not None:
xs.name = name
xs.intensity = probe.pp.intensity
xs.sample_broadening = probe.pp.sample_broadening
xs.theta_offset = probe.pp.theta_offset
xs.background = probe.pp.background

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probe.shared_beam() does this.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I tried the below code, but the parameter name tags are lost - e.g. if name="ni58" that does not get passed to the parameter names. When looking at probe.shared_beam() the names are hard coded and have no scope to pull name from the probe class - or even accept an optional name from the call.

probe = PolarizedNeutronProbe(cross_sections, Aguide=270, H=field, name=name)

# looking at shared_beam, it doesn't set by default intensity etc. 
# to the values originally passed when the probe object is created.
# Instead will pass them manually from probe.
probe.shared_beam(intensity=probe.pp.intensity.value, 
                  background=probe.pp.background.value, 
                  back_absorption=probe.pp.back_absorption.value, 
                  theta_offset=probe.pp.theta_offset.value, 
                  sample_broadening=probe.pp.sample_broadening.value)

If you would like me to use shared beam I would suggest changing probe.shared_beam() to something like:

    def shared_beam(self, intensity=None, background=None, back_absorption=None, theta_offset=None, sample_broadening=None, name=None):
        """
        Share beam parameters across all segments.

        New parameters are created for *intensity*, *background*,
        *theta_offset*, *sample_broadening* and *back_absorption*
        and assigned to the all segments.  These can be replaced
        with an explicit parameter in an individual segment if that
        parameter is independent.
        """
        if not intensity:
             intensity = self.probes[0].intensity
        ...

        intensity = Parameter.default(intensity, name=f"intensity {name}")
        background = Parameter.default(background, name=f"background {name}", limits=[0, None])
        back_absorption = Parameter.default(back_absorption, name=f"back_absorption {name}", limits=[0, 1])
        theta_offset = Parameter.default(theta_offset, name=f"theta_offset {name}")
        sample_broadening = Parameter.default(sample_broadening, name=f"sample_broadening {name}", limits=[None, None])
        for p in self.probes:
            p.intensity = intensity
            p.background = background
            p.back_absorption = back_absorption
            p.theta_offset = theta_offset
            p.sample_broadening = sample_broadening

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about adding the following to probe.py:

BEAM_TAGS = ('inst', 'nuisance')
SHARED_BEAM_KW = {'intensity', 'background', 'back_absorption', 'theta_offset', 'sample_broadening'}
...
def shared_beam(self, ...):
    ...
    intensity = Parameter.default(intensity, name="intensity {self.name}", tags=BEAM_TAGS)
    ...

self.name defaults to mm.name if no name was given to PolarizedNeutronProbe.

Change PROBE_KW to create a set {...} instead of a tuple (...), and update every Parameter.default to include tags=BEAM_TAGS.

In load_probe_polref():

beam_kw = {k: v for k, v in kw.items() if k in SHARED_BEAM_KW}
probe_kw = {k: v for k, v in kw.items() if k not in SHARED_BEAM_KW}
...
for data in files.values():
    TOF_loader(..., **probe_kw)
...
probe.shared_beam(**beam_kw)

In practice it seems that sample_broadening, back_absorption come from the sample and should always be shared.

Theta offset should be a property of alignment, so it should probably be shared. (If alignment changes with motion, then reflectivity curves are going to be hard to fit).

Intensity might be hard to normalize and background might depend on polarization state so I can imagine them being independent.

Maybe we could do partial sharing by calling with unshared parameters as None. For example, shared_beam(background=None) would share everything exception background.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probe.shared_beam() should work after #388

I've set it up so that the defaults come from the first probe, so any beam parameter keywords passed through **kw will be respected in the shared beam.

There is no simple way to "unshare" beam parameters. You may want to support load_probe_polref(..., shared_beam=False).


probe.pp.intensity.name = f"intensity {name}"
probe.pp.background.name = f"background {name}"
probe.pp.sample_broadening.name = f"sample_broadening {name}"
probe.pp.theta_offset.name = f"theta_offset {name}"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These parameters are already tagged with {name}.

If you switch to probe.shared_beam above it will create new parameters with untagged names. Please address #385 and remove the tagging here.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Explicit renaming (above) and tagging (below) won't be necessary after #387 and #388.


probe.pp.intensity.tags = ["inst", "nuisance"]
probe.pp.background.tags = ["inst", "nuisance"]
probe.pp.sample_broadening.tags = ["inst", "nuisance"]
probe.pp.theta_offset.tags = ["inst", "nuisance"]

return probe


# TODO: Add simulation wrapper based on the loader above for simulating POLREF data.
Loading
Loading