Time-Series Correction (Atmosphere - Ionosphere) and Delaunay Network

[giorgoskouroupis]

Hello @scottstanie and everyone!

I am testing dolphin, with a series of slcs downloaded from ASF and pre-processed with ISCE2.
Everything seems good after some tweaking.

However, I have seen that the network of interferomgrams is the "Single-Reference". Are the there any plans to do the Delaunay or sequential networks? (originated in the miaplpy

Also, even though i see in the config file the inputs for atmospheric and ionospheric corrections, i could not manage to run it successfully.
i. Are they implemented?
ii. if yes, are there any instructions, as to what should be the input data and in what format?

Thank you in advance for your time :)

P.S. i did a search for answers within the project but i could not find any results, if I missed them, please guide me to them.

[scottstanie]

Hi,
For (1), yes there are definitely other interferogram network options. I think I may change the default of single-reference relative to index 0, since I mostly use max_bandwidth: 3, which forms the nearest three neighbor interferograms in time.

The sample config has the interferogram_network section. If you only want nearest-3, you can delete the reference_idx: and make the next line max_bandwidth: 3.
I don't have any Dealaunay in the perpendicular-baseline/temporal-baseline space, since most of the work has been using Sentinel-1 (where the Delaunay graph basically degenerates into a nearest-N graph, since the orbital tubes are so small).

2. We do have a preliminary corrections workflow to be run after the DisplacementWorkflow, but the application is somewhat limited- The existing weather-model based troposphere correction was split off into this repo: https://github.com/opera-adt/tropo (which will be completed in the next 1-2 months), and the ionosphere correction is a range-only based correction for now. We don't have any split-spectrum, or any attempts to solve for burst-to-burst discontinuities based on the ionosphere.

Which type of correction were you hoping to apply?

[giorgoskouroupis]

Dear @scottstanie,

Thank you for your quick reply.

Regarding (1):
Understood — I see your point. I focused on the Single-Reference and Delaunay networks, especially since in [MiaplPy - Miami Phase-Linking - Fig. 3], they perform better in mixed urban/vegetated areas with seasonal decorrelation.
I think I can approach similar results using a N=3–6 nearest-neighbor (NN) graph — or perhaps by eventually constructing a Delaunay network.
→ Would it be better, instead, to use the "indexes" format for defining a manual-index network, i.e., a list of (ref_idx, sec_idx) tuples specifying the interferograms to form, assuming I have the interferogram pairs that would form the Delaunay Network - for comparison?

Regarding (2):
My main concern is Tropospheric corrections, since I’m testing over northern Europe where ionospheric effects are less significant (or at least I’m avoiding them for now, as I found the split-spectrum results from ISCE2 produced quite noisy interferograms).

So I was considering: why not plug the Dolphin results into MintPy time-series correction pipeline, to benefit from its unwrapping-based corrections as well? → Are there any guidelines on how to make Dolphin output compatible with MintPy?

For unwrapping, I still use snaphu by default — do you have any recommendation between the available options: ['snaphu', 'icu', 'phass', 'spurt', 'whirlwind']?

P.S.
In my case, working with an ISCE2 stack, if I specify bounds under "output_options", PS masking fails and returns an all-black result — which kind of makes sense, as my stack isn’t geocoded.
Still under "output_options", if I leave "bounds_epsg" empty, the code crashes, since it’s a required input. But since I don’t have an actual EPSG code, I’m forced to insert a dummy 4326 to keep it running. This makes the final results appear geocoded, which led to confusion when trying to integrate them into MintPy (still not being able to yet).

I assume this issue stems from the fact that the OPERA CSLC data is co-registered — but is there a way to avoid have them as geocoded so that false geocoding doesn't propagate downstream?

Thank you again in advance for your time :)
Best

[scottstanie]

1. If you do want to specify the exact network that would come from delaunay triangulation, yes I think doing those indexes would be the quickest way. I think if you did, say, nearest 5, a lot of areas of interest would come out with the same quality results as a delaunay triangulation.

2. Yep Dolphin can plug in to Mintpy - https://github.com/OPERA-Cal-Val/calval-DISP/blob/c1589d1cbbbe1d858d8d73d0225a9a3429f4dbbc/dolphin_PSTworkflow/prep_mintpy_dolphin_timeseries.py#L531 the OPERA project science team was using this to set up the mintpy-compatible hdf5 files. With more work on dolphin (and probably on mintpy), we could probably have outputs directly get saved to an HDF5 file, so it would be just adding more mintpy-requested metadata. For now, a data reformatting is needed.

For unwrapping: I use both snaphu and spurt regularly. Spurt can look better on sparesely-coherence areas (i.e. were maybe 5-15% of the pixels are "good pixels"). whirlwind is promising, but needs more work in the future.

thanks for poiting out the bounds. I believe it will not error if you delete the whole line, rather than keep bounds_epsg: (which looks like passing None to it. But I made the fix here anyway. It wasn't related to any assumptions about requiring geocoding or not- both radar and geo coordinates should work

[giorgoskouroupis]

Thank you for your reply, and your PR.
I will pull to try, and i will also try the spurt to compare with snaphu.

I have a couple of remaining questions, which are not directly related to this discussion. Please, allow to ask them here, but if you prefer to move this to a different discussion no problem by me and i can mark this discussion as "Answered".

1. Just for Validation. By applying the --strides will it will do the multilooking, correct? So the input should be "full-resolution".
2. I cannot tell if the process runs on the GPU or not. I have installed the cuda tookit fro here. Then I enable it on the config file. Then with the watch nvidia-smi I can see that the dolphin command put data on the GPU but the GPU usage remains at 0%. Is this correct, or half-done?
3. I am interested in doing a large-scale analysis, and I see that the further I am away from my point-of-reference the more noisy the time-series becomes. Are there any strategies on this point?

Best,
Giorgos

[giorgoskouroupis]

Hello again @scottstanie :)
after some digging and playing around, i saw that the use of SPURT disregards the initial network formation to apply the time-space (hop3 -delaunay) network.
on an other note, in unwrapping.py the if statement will always result in False as the comparison is between a tuple (as per the return of io.get_raster_xysize) and a path. to make it work on my side i had to do as
if io.get_raster_xysize(mask_file) == io.get_raster_xysize(match_file):
to really evaluate the size of the mask (which needs to be GDAL readable)

unless this was not the intended use, please advise.
thank you for your time
best
Giorgos

[scottstanie]

Thank you again for the bug spotting! Fixed here #600

As for the network disregard- Sorry I did not remember that earlier to mention.
That portion of dolphin (where I override whatever is set, while attempting to be loud and clear in the logging) is addressing 2 issues:

1. The temporal unwrapping step can only work with planar graphs, so certain arbitrary unwrapping networks would fail the MCF check. This part I think does not confuse you, but I believe it would be confusing for a general user trying to do 3d unwrapping.
2. @piyushrpt wrote spurt to work with the nearest-3 configuration, as that was the most common working example from their processing experience. He did that by accepting the single-reference phase linking output, then forming interferograms within spurt.
   I tried to get a version where we pass arbitrary interferograms formed in dolphin into the spurt functions, but I was doing something slightly wrong and it never ended up working. This is not an inherent limitation to dolphin or spurt (as the original EMCF use Delaunay); I just had not got back around to making the changes needed to both dolphin and spurt.

[giorgoskouroupis]

Thank you for your reply, and for your quick fix :)

I understand the planarity criterion in terms of computation efficiency. Then, I did not think of the SPURT's network formation overriding as a drawback at all.

I was mostly wondering, since the SPURT will disregard the original network and takes the phase linking SLC results directly as inputs (if not - did I miss something here?), then why bother running the single reference interferograms? Why not avoid the single reference completely , compute the required interferogram network within SPURT, then "push" the interferograms into the "interferograms" folder, and also calculate the correlation for each pair. In this way there would be a good match between the datasets of wrapped-unwrapped interferograms.

[Karthi-Wolf]

Hi, @giorgoskouroupis , Did you manage to run mintpy with the dolphin products?