Dirk, Here is a revised flowgraph, slightly tweaked to reduce the computational cost of the rotator and the filters.
I wasn't intending to tweak the flowgraph, but today I decided to try with a correlation filter with -5 kHz to +5 kHz chirp taps. That method turned out to be inferior to the PLL, so I didn't leave it in the flowgraph. Regards, Andy On Fri, Mar 17, 2017 at 11:46 AM, Andy Walls <[email protected]> wrote: > On Thu, 2017-03-16 at 21:02 -0400, Andy Walls wrote: >> Hi Dirk, >> >> Thanks. >> >> Try the attached flowgraph. The pulse indicator output is a little crude >> at the moment (it needs some latching persistence for a human to read), >> but it does the job for a proof of concept. >> >> Note that in the future, you really do want to tune such the the SDR's center >> freq spike is not in the band of interest. That will prevent false lock on >> the >> DC spike by the PLL. >> >> Regards, >> Andy > > I should also mention a few things: > > 1. You could handle multiple channels in one of at least two ways: > > a. A polyphase channelizer with each channel having a PLL and Moving > average filter instance. (This can get CPU intensive.) Or... > > b. Scanning by stepping through and dwelling for a bit on your 10 kHz > channels of interest. > > > 2. You can get better sensitivity the the PLL+Moving average correlation > solution by going to a 2 stage process: > > a. Use the PLL+Moving Average filter to initially acquire some pulses. > > b. Once you find some pulses, extract what frequency they were at from > the PLL's loop filter state at the time it was locked on the pulse. You > can then use this exact frequency information for a dedicated > correlation filter to pull even weaker pulses out of the noise, and > maintain track. You'll have to use the magnitude of the correlation and > not just the real part (because you'll have an arbitrary phase shift, > since a straight correlation filter is a phase offset detector and not > phase tracker like a PLL). > > Regards, > Andy > > >> On Thu, Mar 16, 2017 at 7:12 PM, Dirk Gorissen <[email protected]> wrote: >> > Hi Andy, >> > >> > Very quickly collected some data off a different Tx. First at very >> > close range, moving away, and then back again. It was quick and >> > antenna was close to computer so perhaps quite noisy but should >> > definitely have some strong pulses: >> > >> > https://drive.google.com/open?id=0B5dKo9igl8W4WkRkUGQzcVJsQnc >> > >> > Havent managed to look at the pll stuff yet. Aim to do so this weekend. >> > Cheers >> > Dirk >> > >> > On 16 March 2017 at 07:22, Dirk Gorissen <[email protected]> wrote: >> >> Mmm, thats odd. Those settings should be correct, maybe something went >> >> wrong with the recording :( I'll try to check in between things at >> >> work today else will do so tonight. >> >> Thanks for taking the time to look though. >> >> >> >> On 16 March 2017 at 00:39, Andy Walls <[email protected]> wrote: >> >>> Hi Dirk: >> >>> >> >>> In the IQ data file you provided I can seem to find any pulses of the >> >>> nominal 10 msec length, no matter how I filter and rotate the >> >>> spectrum. >> >>> There is a lot of EMI, which looks like the intermodulation products >> >>> of a continuously on guy who is drifting/chirping down in frequency. >> >>> >> >>> So could you please confirm or clarify the following: >> >>> >> >>> 1. The format of the binary data file. I am assuming 32 bit float I >> >>> and 32 bit float Q pairs, as that is what the GNURadio file sink would >> >>> normally create. >> >>> 2. The sample rate. I am assuming 1 Msps. >> >>> 3. The center freq. I am assuming 150.22 MHz. >> >>> 4. The pulse duration. I am assuming 10 milliseconds. >> >>> 5. At what frequency offset, from the center frequency, should the >> >>> pulse be at? I'm assuming somewhere withing +/- 5 kHz of the center >> >>> spike, but there are at least two EMI spikes in that range. >> >>> >> >>> Thanks. >> >>> >> >>> -Andy >> >>> >> >>> On Wed, Mar 15, 2017 at 5:23 AM, Dirk Gorissen <[email protected]> >> >>> wrote: >> >>>> Hi Andy, Marcus, >> >>>> >> >>>> Thanks very much for taking the time to think about this. >> >>>> >> >>>> Just to answer your questions: >> >>>> >> >>>>>Dirk's problem was the low SNR in his recording, so he'd like to take >> >>>>>the shape of his pulse into consideration, but he doesn't know the >> >>>>>spectral position of the same – Dirk, can you confirm? >> >>>> >> >>>> Yes, Im dealing with a very weak signal against quite a bit of >> >>>> background noise so the more prior knowledge I can leverage the >> >>>> better. >> >>>> >> >>>>>Dirk, thinking about this, the relation of the spectral uncertainty to >> >>>>>the bandwidth of the pulse would be interesting – can you refresh our >> >>>>>memory? I think the pulse was 10ms long (so, pretty long) but I forgot >> >>>>>how it looked like. >> >>>> >> >>>> Yes, 10ms long (this is pretty exact), occurring every 1.5 seconds, in >> >>>> the 150MHz range. My input stream is coming from an SDR. >> >>>> As an aside I actually have multiple transmitters each pulsing at >> >>>> slightly different frequencies (e.g., 150.10, 150.22, ...) but Im >> >>>> happy to treat them independently so you can ignore that and assume >> >>>> there is just one. >> >>>> >> >>>> You can see pictures of the pulse (taken a while back, for a specific >> >>>> tx frequency) here: https://goo.gl/photos/Y3Ea6kJo1ewrcDYM8 >> >>>> >> >>>> Note though that this is taken with the transmitter very close to the >> >>>> antenna so the signal is unrealistically strong. So its just to >> >>>> illustrate. >> >>>> >> >>>> I also quickly captured a file with raw IQ samples (File sink) in case >> >>>> anybody is interested. It starts with the transmitter very close to >> >>>> antenna, moves progressively further until out of range and then back. >> >>>> Its only about a minute or two tops but at 1msps it ended >> >>>> up as 813 MB though. >> >>>> >> >>>> https://drive.google.com/drive/folders/0B5dKo9igl8W4dWpiMmlEYWx3b0k?usp=sharing >> >>>> >> >>>>>What you really care about is >> >>>>>that there's a *tone* for about 8ms < t < 12ms where there is none else >> >>>>>(to rule out detection of spurs and other interference), is that right? >> >>>> >> >>>> Yes. The "tone" happens every 1.5 seconds and I want to catch as many >> >>>> of those occurrences as possible as the receiver moves through space. >> >>>> So for every 1.5 seconds I need to make a decision: was there one, or >> >>>> not. >> >>>> >> >>>> Note that the receiver is moving. So perhaps initially I may be well >> >>>> out of range of the signal and get nothing. But as I move closer as >> >>>> some point I will start picking up those pulses. The earlier and more >> >>>> reliably I can pick them up the better. >> >>>> >> >>>> I actually did spend some time looking at PLLs & the gnu radio block >> >>>> at some point. However I readily admit to being a software person not >> >>>> a DSP/Radio person. I have the day job to deal with today but I will >> >>>> have a study of the PLL block given Andy's tips and report back. >> >>>> >> >>>> Cheers >> >>>> Dirk >> >>>> >> >>>> >> >>>> >> >>>> >> >>>> >> >>>> On 14 March 2017 at 16:37, Andy Walls <[email protected]> >> >>>> wrote: >> >>>>> On Tue, 2017-03-14 at 12:00 -0400, [email protected] >> >>>>> wrote: >> >>>>>> Date: Tue, 14 Mar 2017 15:49:44 +0100 >> >>>>>> From: Marcus M?ller <[email protected]> >> >>>>>> To: [email protected] >> >>>>>> Subject: Re: [Discuss-gnuradio] fast parallel filtering >> >>>>> >> >>>>>> Hi Andy, >> >>>>>> >> >>>>>> Dirk's problem was the low SNR in his recording, so he'd like to take >> >>>>>> the shape of his pulse into consideration, but he doesn't know the >> >>>>>> spectral position of the same ? Dirk, can you confirm? >> >>>>> >> >>>>> Ah, OK. >> >>>>> >> >>>>> >> >>>>>> Dirk, thinking about this, the relation of the spectral uncertainty to >> >>>>>> the bandwidth of the pulse would be interesting ? can you refresh our >> >>>>>> memory? I think the pulse was 10ms long (so, pretty long) but I forgot >> >>>>>> how it looked like. >> >>>>>> >> >>>>>> Having slept about this a couple of times: What you really care about >> >>>>>> is >> >>>>>> that there's a *tone* for about 8ms < t < 12ms where there is none >> >>>>>> else >> >>>>>> (to rule out detection of spurs and other interference), is that >> >>>>>> right? >> >>>>>> >> >>>>>> Maybe we've been focussing too much on filter-based detection. What if >> >>>>>> we just *use* that feature of the signal being periodic for a duration >> >>>>>> of t, and not else? A PLL would be able to "lock" on the tone (much >> >>>>>> like >> >>>>>> an FM Radio with a PLL will lock on the station's carrier), and if we >> >>>>>> observe the phase error being limited for t, we can derive there was a >> >>>>>> pulse. >> >>>>>> >> >>>>>> Andy, does that sound like a feasible approach? >> >>>>> >> >>>>> Yes. With the added benefit of GNURadio's carrier tracking PLL >> >>>>> actually >> >>>>> performing part of the correlation operation for you, if it works out. >> >>>>> >> >>>>> So use the carrier_tracking_pll block. If it locks onto a good signal, >> >>>>> it will mix it down to DC. Then all you need is an averaging filter, >> >>>>> like the single pole IIR filter block or the moving average filter >> >>>>> block, operating on the real part of that output. That will act as a >> >>>>> lock detector, and, I think, also completes a correlation operation, if >> >>>>> you use a non-normalized moving average filter (since the carrier >> >>>>> tracking PLL block gives you a point by point complex multiply with the >> >>>>> conjugate of the complex carrier tone). >> >>>>> >> >>>>> You'll want to set the PLL allowed min and max frequency bounds >> >>>>> properly; be careful since the input arguments have tricky units. >> >>>>> >> >>>>> You'll also want the loop bandwidth set pretty wide, since you want to >> >>>>> lock-in rapidly. >> >>>>> >> >>>>> Also, if I'm thinking about this properly: >> >>>>> To get faster lock in, you may want your frequency range of interest to >> >>>>> be somewhere in between Fs/4 and Fs/2; and not near DC. If you're >> >>>>> within the lock-in range of the PLL, you'll lock within 1 cycle. If >> >>>>> you're in the pull-in range of the PLL, it can take many cycles to get >> >>>>> into lock. >> >>>>> >> >>>>> Regards, >> >>>>> Andy >> >>>>> >> >>>>>> Cheers, >> >>>>>> >> >>>>>> Marcus >> >>>>>> >> >>>>>> >> >>>>>> On 14.03.2017 14:18, Andy Walls wrote: >> >>>>>> > If there is no information on the signal, why not just do a straight >> >>>>>> > energy detector: >> >>>>>> > >> >>>>>> > source -> complex bandpass filter -> complex to mag -> burst/energy >> >>>>>> detector -> QT Time sink (triggering on start of burst tag) >> >>>>>> > >> >>>>>> > The complex bandpass filter just has to catch all the frequencies of >> >>>>>> > interest. (A complex bandpass filter does not have a symmetrical >> >>>>>> image >> >>>>>> > around DC.) >> >>>>>> > >> >>>>>> > The burst/energy detector has to detect some minimum number of time >> >>>>>> > domain samples contiguously above some noise/signal threshold that >> >>>>>> you >> >>>>>> > set, and tag the pulse. It also has to detect when the time domain >> >>>>>> > samples fall below the threshold. The burst/energy detector can >> >>>>>> work >> >>>>>> > with a preset noise/signal threshold, or you could periodically >> >>>>>> > re-estimate that threshold. >> >>>>>> > >> >>>>>> > GNURadio does not have a stock block that does burst/energy >> >>>>>> detection, >> >>>>>> > so you'll have to find one on the 'Net or write one yourself. >> >>>>>> > >> >>>>>> > -Andy >> >>>>>> > >> >>>>>> > >> >>>>>> > Dirk Gorissen wrote: >> >>>>>> >> Hi Martin, >> >>>>>> >> >> >>>>>> >> The aim is to check for the existence of a 10ms pulse in the >> >>>>>> incoming >> >>>>>> >> radio signal (from an sdr). The thing is I only know the frequency >> >>>>>> of >> >>>>>> >> the pulse to within ~1khz. >> >>>>>> >> >> >>>>>> >> So a single filter in my case is: generate a synthetic pulse >> >>>>>> (complex >> >>>>>> >> sin wave) of the same length and with a frequency f. Then take the >> >>>>>> >> reverse of the complex conjugate of this pulse to give me the taps >> >>>>>> for >> >>>>>> >> a FIR filter. >> >>>>>> >> >> >>>>>> >> Repeat the above for each frequency I want to check. E.g., 10 >> >>>>>> filters >> >>>>>> >> each 100Hz apart. >> >>>>>> >> Then I just take the magnitude of each filter output and push >> >>>>>> through >> >>>>>> >> a Max to get my final correlations. >> >>>>>> >> >> >>>>>> >> I can come up with an algorithm that tries to be clever and with >> >>>>>> some >> >>>>>> >> accounting tries to find what frequency matters most but I wouldnt >> >>>>>> be >> >>>>>> >> surprised if there is some theory you can use to do this more >> >>>>>> >> efficiently or even in a single shot. But this is where Im unsure. >> >>>>>> >> >> >>>>>> >> Cheers >> >>>>>> >> Dirk >> >>>>>> >> >> >>>>>> >> >> >>>>>> >> On 14 March 2017 at 01:09, Martin Braun <address@hidden> wrote: >> >>>>>> >>> How related are those filters? Is this a candidate for polyphase >> >>>>>> DSP? >> >>>>>> >>> >> >>>>>> >>> -- M >> >>>>>> >>> >> >>>>>> >>> On 03/11/2017 02:01 PM, Dirk Gorissen wrote: >> >>>>>> >>>> Hi Marcus, >> >>>>>> >>>> >> >>>>>> >>>> Sorry, I should have clarified. You may recall an earlier thread >> >>>>>> from >> >>>>>> >>>> mine where Im looking to pick out a short pulse from background >> >>>>>> noise >> >>>>>> >>>> but I dont know the exact frequency of the pulse. Thought I >> >>>>>> would >> >>>>>> >>>> start a new thread with a clear, specific question. >> >>>>>> >>>> >> >>>>>> >>>> There is an uncertainty of +/- 1khz. So I can define multiple >> >>>>>> filters, >> >>>>>> >>>> correlating for different pulse frequencies across the 1khz >> >>>>>> range. I >> >>>>>> >>>> can implement this with different filters running in parallel >> >>>>>> but I >> >>>>>> >>>> was looking for a more flexible / efficient way. >> >>>>>> >>>> >> >>>>>> >>>> If you think this is out of scope for this list no problem at >> >>>>>> all, >> >>>>>> >>>> just let me know. >> >>>>>> >>>> >> >>>>>> >>>> Cheers >> >>>>>> >>>> Dirk >> >>>>>> >>>> >> >>>>>> >>>>> On 11 March 2017 at 20:02, Marcus M?ller <address@hidden> wrote: >> >>>>>> >>>>>> Hi Dirk, >> >>>>>> >>>>>> >> >>>>>> >>>>>> this is more of a general DSP question than a GNU Radio >> >>>>>> question: >> >>>>>> >>>>>> >> >>>>>> >>>>>> How do these filters relate to each other? >> >>>>>> >>>>>> >> >>>>>> >>>>>> My gut feeling is that this gets a lot easier as soon as you >> >>>>>> tell us why >> >>>>>> >>>>>> you're doing this, i.e. for what purpose :) >> >>>>>> >>>>>> >> >>>>>> >>>>>> Best regards, >> >>>>>> >>>>>> Marcus >> >>>>>> >>>>>> >> >>>>>> >>>>>> On 11.03.2017 19:28, Dirk Gorissen wrote: >> >>>>>> >>>>>>> Hello all, >> >>>>>> >>>>>>> >> >>>>>> >>>>>>> Given a stream of samples I would like to apply n slightly >> >>>>>> different >> >>>>>> >>>>>>> filters to it with n being able to be chosen at runtime. Then >> >>>>>> combine >> >>>>>> >>>>>>> the results back to a single stream. >> >>>>>> >>>>>>> >> >>>>>> >>>>>>> As a test I built a flowgraph with the following chains in >> >>>>>> parallel for n >> >>>>>> >>>>>>> = 6 >> >>>>>> >>>>>>> >> >>>>>> >>>>>>> | -> decimating fir filter 1 -> complex to >> >>>>>> mag -> | >> >>>>>> >>>>>>> stream -> | -> decimating fir filter 2 -> complex to mag -> >> >>>>>> | -> Max >> >>>>>> >>>>>>> -> ... >> >>>>>> >>>>>>> | .... >> >>>>>> >>>>>>> | >> >>>>>> >>>>>>> | -> decimating fir filter n -> complex to >> >>>>>> mag -> | >> >>>>>> >>>>>>> >> >>>>>> >>>>>>> So the same stream is sent to each chain (decimation is 1) and >> >>>>>> the >> >>>>>> >>>>>>> output of each chain is pushed through a big Max block with 6 >> >>>>>> inputs. >> >>>>>> >>>>>>> >> >>>>>> >>>>>>> This works but not particularly elegant and a bit annoying to >> >>>>>> change >> >>>>>> >>>>>>> if I suddenly decide I want to change n. In particular it also >> >>>>>> does >> >>>>>> >>>>>>> not seem computationally efficient. >> >>>>>> >>>>>>> >> >>>>>> >>>>>>> What I would like is to replace the above by a single block >> >>>>>> that >> >>>>>> >>>>>>> >> >>>>>> >>>>>>> - replicates the input n times >> >>>>>> >>>>>>> - applies each filter on each replica >> >>>>>> >>>>>>> - combines the output again to a single stream >> >>>>>> >>>>>>> - have a tunable n parameter >> >>>>>> >>>>>>> - is fast >> >>>>>> >>>>>>> >> >>>>>> >>>>>>> I did this with an Embedded python block doing essentially >> >>>>>> this: >> >>>>>> >>>>>>> >> >>>>>> >>>>>>> for i in range(n): >> >>>>>> >>>>>>> out[i] = scipy.signal.lfilter(taps[i], 1, input) >> >>>>>> >>>>>>> >> >>>>>> >>>>>>> This is using exactly the same taps as in the chain case. This >> >>>>>> works >> >>>>>> >>>>>>> but the output is different and worse than what I get with the >> >>>>>> >>>>>>> separate chains. As a test, instead of lfilter I tried: >> >>>>>> >>>>>>> >> >>>>>> >>>>>>> >> >>>>>> gnuradio.filter.fir_filter_ccc(1,taps[i]).work(input[0],output) >> >>>>>> >>>>>>> >> >>>>>> >>>>>>> Thinking perhaps that is a closer replica. But couldnt get it >> >>>>>> to work.. >> >>>>>> >>>>>>> >> >>>>>> >>>>>>> I suspect there should be an easy / natural way of doing this >> >>>>>> in >> >>>>>> >>>>>>> gnuradio. Looked at the filter bank / channelliser blocks but >> >>>>>> failed >> >>>>>> >>>>>>> to get anywhere. >> >>>>>> >>>>>>> >> >>>>>> >>>>>>> So what is the best way forward to do this? >> >>>>>> >>>>>>> >> >>>>>> >>>>>>> Many thanks >> >>>>>> >>>>>>> Dirk >> >>>>>> >>>>>>> >> >>>>>> >>>>>>> _______________________________________________ >> >>>>>> >>>>>>> Discuss-gnuradio mailing list >> >>>>>> >>>>>>> address@hidden >> >>>>>> >>>>>>> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio >> >>>>>> >>>>>> >> >>>>>> >>>>>> _______________________________________________ >> >>>>>> >>>>>> Discuss-gnuradio mailing list >> >>>>>> >>>>>> address@hidden >> >>>>>> >>>>>> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio >> >>>>>> >>>>> >> >>>>>> >>>>> >> >>>>>> >>>>> -- >> >>>>>> >>>>> _________________________________________ >> >>>>>> >>>>> Dr. Dirk Gorissen >> >>>>>> >>>>> Mob: +44-7763-806-809 >> >>>>>> >>>>> Web: dirkgorissen.com >> >>>>>> >>>>> Skype: dirk.gorissen >> >>>>>> >>>>> Twitter : twitter.com/dirkgor >> >>>>>> >>>>> LinkedIn: linkedin.com/in/dirkgorissen >> >>>>>> >>>> >> >>>>>> >>>> >> >>>>>> >>> >> >>>>>> >>> _______________________________________________ >> >>>>>> >>> Discuss-gnuradio mailing list >> >>>>>> >>> address@hidden >> >>>>>> >>> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio >> >>>>>> >> >> >>>>>> >> >> >>>>>> >> -- >> >>>>>> >> _________________________________________ >> >>>>>> >> Dr. Dirk Gorissen >> >>>>>> >> Mob: +44-7763-806-809 >> >>>>>> >> Web: dirkgorissen.com >> >>>>>> >> Skype: dirk.gorissen >> >>>>>> >> Twitter : twitter.com/dirkgor >> >>>>>> >> LinkedIn: linkedin.com/in/dirkgorissen >> >>>>>> > >> >>>>>> > >> >>>>>> > _______________________________________________ >> >>>>>> > Discuss-gnuradio mailing list >> >>>>>> > [email protected] >> >>>>>> > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio >> >>>>> >> >>>>> >> >>>> >> >>>> >> >>>> >> >>>> -- >> >>>> _________________________________________ >> >>>> Dr. Dirk Gorissen >> >>>> Mob: +44-7763-806-809 >> >>>> Web: dirkgorissen.com >> >>>> Skype: dirk.gorissen >> >>>> Twitter : twitter.com/dirkgor >> >>>> LinkedIn: linkedin.com/in/dirkgorissen >> >> >> >> >> >> >> >> -- >> >> _________________________________________ >> >> Dr. Dirk Gorissen >> >> Mob: +44-7763-806-809 >> >> Web: dirkgorissen.com >> >> Skype: dirk.gorissen >> >> Twitter : twitter.com/dirkgor >> >> LinkedIn: linkedin.com/in/dirkgorissen >> > >> > >> > >> > -- >> > _________________________________________ >> > Dr. Dirk Gorissen >> > Mob: +44-7763-806-809 >> > Web: dirkgorissen.com >> > Skype: dirk.gorissen >> > Twitter : twitter.com/dirkgor >> > LinkedIn: linkedin.com/in/dirkgorissen > >
implant_pulse_detect.grc
Description: Binary data
_______________________________________________ Discuss-gnuradio mailing list [email protected] https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
