ENH: sosfreqz #6328

larsoner · 2016-06-28T20:33:10Z

I had no idea how close to mergeable @WarrenWeckesser's #4465 was. This should be ready for review/merge, I just rebased and tweaked the release notes.

e-q · 2016-06-28T22:47:01Z

Thanks for picking this up! I got sucked down a rabbit hole when trying to make this part of gh-6059, which I will now ramble on about:

Basically, I was thinking it would be good if proper SOS and ZPK frequency responses could just live behind (d)freqresp without needing to create more public functions. This led me to look into creating the SOS LTI objects classes which we definitely want; however, this breaks all of the functions with call signatures that count arguments to determine the filter representation.

I.e., we specify the SOS arrays as arrays of shape (N, 6), but functions like freqresp use their arguments to create an LTI object via lti(*args) which means lti(*sos) can look like a TF, zpk, or SS depending on N. So, we need to either deprecate those call signatures, or find a way to determine the difference between a (2,6) SOS array and length six (b, a) coefficients.

All this is to say: we probably do indeed need this, and the more accurate zpk version, to be public. We should also come up with a solution to the SOS LTI object issue sooner rather than later.

larsoner · 2016-06-29T02:38:57Z

we probably do indeed need this, and the more accurate zpk version, to be public

Maybe just for now we do the SOS, and then someone else can make public the improved ZPK one?

We should also come up with a solution to the SOS LTI object issue sooner rather than later.

Yeah I would like to see that, too, but I worry about holding this one up to wait on it. Are you okay going forward with sosfreqz for now?

e-q · 2016-06-29T03:18:47Z

Yeah I would like to see that, too, but I worry about holding this one up to wait on it. Are you okay going forward with sosfreqz for now?

Sorry for being unclear, this is what I was getting at. I agree that this PR should happen sooner rather than later.

WarrenWeckesser · 2016-06-29T14:31:30Z

doc/release/0.19.0-notes.rst

+`scipy.signal` improvements
+---------------------------
+
+The funtion `scipy.signal.sosfreqz` was added to compute the frequency


WarrenWeckesser · 2016-06-29T14:53:12Z

Eric, thanks for picking this up. The changes look good; I made a few suggestions in the code.

My PR was pretty close to ready. The main issue (which I agree with) was the API. I copied it from freqz just to be consistent, but it isn't a great starting point. (worN is an awkward argument name, and if not for maintaining backwards compatibility, I'd remove the plot argument from freqz.) Alternatives have been discussed elsewhere, but their implementation is not imminent, so perhaps it would be best to get sosfreqz in now, knowing that it might become obsolete with the next generation of the signal API. The perfect is the enemy of the good and all that...

larsoner · 2016-06-29T17:34:32Z

@WarrenWeckesser any idea how your changes could have broken some scipy.special tests for mpmath?

larsoner · 2016-06-29T17:35:27Z

I removed the plot argument since I don't think it should really be there (even in freqz), but I kept the other arguments named the same so there is some compatibility with the existing function.

person142 · 2016-06-29T19:56:16Z

scipy/signal/tests/test_filter_design.py

+        N = 500
+        order = 25
+        Wn = 0.15
+        mpmath.mp.dps = 80


@Eric89GXL: I think the test failure comes from increasing the dps here. mpmath uses a global precision and running test_hyp0f1_gh5764 with a dps of 80 fails. IOW test_hyp0f1_gh5764 is bad... but at low precision mpmath happens to be bad in the same way.

I bet restoring the original precision at the end of test_sos_freqz_against_mp will fix the problem.

Ahh I tried to find some global setting but missed this. Thanks for finding it, I'll set it back

larsoner · 2016-06-29T20:29:43Z

@endolith do you want to have a look since you have been involved in SOS?

@rgommers you've been involved with the API discussions, can you live with another function that isn't part of the grand-unified API?

person142 · 2016-06-29T20:30:44Z

scipy/signal/tests/test_filter_design.py

+        Wn = 0.15
+        old_dps = mpmath.mp.dps
+        mpmath.mp.dps = 80
+        try:


FWIW the mpmath.workdps context manager is nice for doing this sort of thing.

Cool, I'll put that in

rgommers · 2016-06-29T21:15:03Z

@rgommers you've been involved with the API discussions, can you live with another function that isn't part of the grand-unified API?

Yes. One more or less is not going to make the difference, and this is functionality that is needed.

endolith · 2016-06-29T23:35:36Z

scipy/signal/filter_design.py

+    a digital filter, compute the frequency response of the system function::
+
+               B0(z)*B1(z)*...*B{n-1}(z)
+        H(z) = -------------------------


How about

B0(z) B1(z) B{n-1}(z) H(z) = ----- * ----- * ... * --------- A0(z) A1(z) A{n-1}(z)

to imply the stages being calculated separately and then combined?

endolith · 2016-06-30T00:01:35Z

Also would it be worth showing a higher-order filter that becomes garbled with freqz, showing them side-by-side to illustrate why sos is beneficial?

endolith · 2016-06-30T00:03:48Z

scipy/signal/filter_design.py

+    >>> sos = signal.ellip(8, 0.5, 60, (0.2, 0.4), btype='bandpass',
+    ...                    output='sos')
+
+    Compute the frequency response on a grid of 1500 points.


Compute the frequency response at 1500 points from DC to Nyquist.?

endolith · 2016-06-30T00:08:48Z

Also would it make sense to have a parameter that makes it output the responses for each stage separately, so you can plot them individually? https://en.wikibooks.org/wiki/Signals_and_Systems/Second_Order_Transfer_Function#Amplitude_Responses

You can break up the sos array and do it yourself, but a function would be more convenient.

This looks fine to me otherwise.

endolith · 2016-06-30T00:11:39Z

scipy/signal/filter_design.py

+
+    sos, n_sections = _validate_sos(sos)
+    if n_sections == 0:
+        raise ValueError('Cannot compute frequencies with no sections')


Should freqz([], [], worN=10) raise a ValueError, too?

Perhaps, but it might be a backward incompatible change...?

endolith · 2016-06-30T00:38:51Z

Suggested changes in endolith@b35fbc6

Mpmath computes the test values in `test_mpmath:test_hyp0f1_gh5764` correctly at small dps, incorrectly at medium dps, and then correctly again at large dps. This sets the dps to be high to avoid confusion. See scipy#6328 (comment).

larsoner · 2016-06-30T13:21:53Z

Thanks @endolith cherry-picked and updated.

Based on the comments I think this should be mergeable. If anyone else wants to look again (or wants me to wait) feel free, otherwise I'll put it in next week.

endolith · 2016-07-01T03:05:54Z

For testing, you could generate some high-order filters as SOS and then check that sosfreqz is within bounds for what the filter should do, like the plots and tests for these:

larsoner · 2016-07-06T19:33:52Z

@endolith commit added, see if you're happy now

Mpmath computes the test values in `test_mpmath:test_hyp0f1_gh5764` correctly at small dps, incorrectly at medium dps, and then correctly again at large dps. This sets the dps to be high to avoid confusion. See scipy#6328 (comment).

endolith · 2016-07-11T23:52:20Z

I started modifying the test cases to make them more extreme (so that sosfreqz could handle them but freqz couldn't) but one of them fails, and I don't know if it's sosfreqz or ellipord or ellip that's actually responsible:

N, Wn = ellipord(0.3, 0.25, .5, 150)
sos = ellip(N, .5, 150, Wn, 'high', output='sos')
w, h = sosfreqz(sos)
dB = 20*np.log10(np.abs(h))
w /= np.pi
plot(w, dB)

equiripple should be below -150 dB, but it's more like -147:

WarrenWeckesser · 2016-07-12T00:54:14Z

@endolith wrote:

I started modifying the test cases to make them more extreme (so that sosfreqz could handle them but freqz couldn't)

That's also what TestSOSFreqz.test_sos_freqz_against_mp() does--it verifies the correctness of an order 25 filter by comparing it to the result computed using mpmath. The mpmath functions are in tests/mpsig.py.

endolith · 2016-07-13T04:35:13Z

anyway these are the checks I made:

        # from cheb2ord
        N, Wn = cheb2ord([0.1, 0.6], [0.2, 0.5], 3, 150)
        sos = cheby2(N, 150, Wn, 'stop', output='sos')
        w, h = sosfreqz(sos)
        dB = 20*np.log10(np.abs(h))
        w /= np.pi
        assert_allclose(dB[w <= 0.1], 0, atol=3.01)
        assert_allclose(dB[w >= 0.6], 0., atol=3.01)
        assert_array_less(dB[(w >= 0.2) & (w <= 0.5)], -149.9)

        # from cheb1ord
        N, Wn = cheb1ord(0.2, 0.3, 1, 150)
        sos = cheby1(N, 1, Wn, 'low', output='sos')
        w, h = sosfreqz(sos)
        dB = 20*np.log10(np.abs(h))
        w /= np.pi
        assert_allclose(dB[w <= 0.2], 0, atol=1.01)
        assert_array_less(dB[w >= 0.3], -149.9)

        # adapted from ellipord
        N, Wn = ellipord(0.3, 0.25, .5, 150)
        sos = ellip(N, .5, 150, Wn, 'high', output='sos')
        w, h = sosfreqz(sos)
        dB = 20*np.log10(np.abs(h))
        w /= np.pi
        assert_allclose(dB[w >= 0.3], 0, atol=.55)
        assert_array_less(dB[(w > 0) & (w <= 0.25)], -149.9)

        # adapted from buttord
        N, Wn = buttord([0.2, 0.5], [0.14, 0.6], 3, 100)
        sos = butter(N, Wn, 'band', output='sos')
        w, h = sosfreqz(sos)
        dB = 20*np.log10(np.abs(h))
        w /= np.pi
        assert_array_less(dB[(w > 0) & (w <= 0.14)], -99.9)
        assert_array_less(dB[w >= 0.6], -99.9)
        assert_allclose(dB[(w >= 0.2) & (w <= 0.5)], 0, atol=3.01)

Change equation format to clarify stages Italicize variables Clarify output complex format Remove obsolete plot= description Increase order of example filter to highlight benefits of SOS Add an example of same-order filter implemented as single stage

larsoner · 2016-07-13T15:22:17Z

Thanks @endolith added your tests (after making the problematic ellipord test use 147 instead of ~149 for the acceptable level), squashed and rebased.

If there are no other comments I'll merge in a couple of days.

larsoner · 2016-07-17T20:57:24Z

Thanks for the reviews

larsoner mentioned this pull request Jun 28, 2016

WIP: signal: Add the function sosfreqz. #4465

Closed

WarrenWeckesser reviewed Jun 29, 2016
View reviewed changes

person142 reviewed Jun 29, 2016
View reviewed changes

rgommers added enhancement A new feature or improvement scipy.signal labels Jun 29, 2016

endolith reviewed Jun 29, 2016
View reviewed changes

endolith reviewed Jun 30, 2016
View reviewed changes

person142 mentioned this pull request Jun 30, 2016

MAINT: special: fix a test for hyp0f1 #6334

Merged

WarrenWeckesser and others added 4 commits July 13, 2016 11:20

ENH: signal: Add the function sosfreqz.

5137b83

FIX: Minor tweaks

f18713f

ENH: Add tests for filter design with SOS

0626be0

larsoner force-pushed the sosfreqz branch from 8c180fe to 0626be0 Compare July 13, 2016 15:20

endolith mentioned this pull request Jul 17, 2016

BUG: Discrepancy in ellipord or ellip #6384

Closed

larsoner merged commit f7deeac into scipy:master Jul 17, 2016

larsoner deleted the sosfreqz branch July 17, 2016 20:57

ev-br added this to the 0.19.0 milestone Jul 22, 2016

Uh oh!

ENH: sosfreqz #6328

ENH: sosfreqz #6328

Uh oh!

Conversation

larsoner commented Jun 28, 2016

Uh oh!

e-q commented Jun 28, 2016

Uh oh!

larsoner commented Jun 29, 2016

Uh oh!

e-q commented Jun 29, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

WarrenWeckesser commented Jun 29, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

larsoner commented Jun 29, 2016

Uh oh!

larsoner commented Jun 29, 2016

Uh oh!

person142 Jun 29, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

larsoner commented Jun 29, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rgommers commented Jun 29, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

endolith commented Jun 30, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

endolith commented Jun 30, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

endolith commented Jun 30, 2016

Uh oh!

larsoner commented Jun 30, 2016

Uh oh!

endolith commented Jul 1, 2016

Uh oh!

larsoner commented Jul 6, 2016

Uh oh!

endolith commented Jul 11, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

WarrenWeckesser commented Jul 12, 2016

Uh oh!

endolith commented Jul 13, 2016

Uh oh!

larsoner commented Jul 13, 2016

Uh oh!

larsoner commented Jul 17, 2016

Uh oh!

Uh oh!

WarrenWeckesser commented Jun 29, 2016 •

edited

Loading

person142 Jun 29, 2016 •

edited

Loading

endolith commented Jun 30, 2016 •

edited

Loading

endolith commented Jun 30, 2016 •

edited

Loading

endolith commented Jul 11, 2016 •

edited

Loading