EachLine only executed after Exec exits?

[xxxserxxx]

It appears that EachLine() is only called after a pipe is closed. This prevents using script to write daemon scripts.

If there's a way to perform the equivalent of:

dbus-monitor --system "interface=org.freedesktop.DBus.Properties" | \
    while read -r line; do
        printf "DBus event: %s\n" "$line"
    done

would it be possible to get an example? If not, can we change the behavior of EachLine() to process each line as it is received, or add an EachLineForever() function?

[bitfield]

Thanks @xxxserxxx! Could you supply a program that demonstrates the behaviour you're talking about?

[xxxserxxx]

I have a bunch of little bash scripts that are long-running processes in the background. One of them watches for DBus events and executes a command when it catches a match:

#!/bin/bash

DE=$(/usr/bin/hcitool con | grep handle | awk '{print $3}' | tr ':' '_')
INTERFACE=org.freedesktop.DBus.Properties
MEMBER=PropertiesChanged
PTH=/org/bluez/hci0/dev_${DE}

/usr/bin/dbus-monitor --system "path='$PTH',interface='$INTERFACE',member='$MEMBER'" |
        while read -r line; do
                echo $line | /usr/bin/grep -q "boolean true"
                if [[ $? -eq 0 ]]; then
                        HA=$(/usr/bin/hcitool con | grep handle | awk '{print $5}')
                        /usr/bin/hcitool lecup --handle ${HA} --min 6 --max 7 --latency 0
                fi
        done

How do you represent any script that does not exit, but continually processes output from a command? Another example would be a script that watches on the output from dmesg and performs some action; or one that tail -F ~/.xsession-error and performs some action whenever a line matches.

[bitfield]

Ah, I see what you're getting at now! This is a good point. As you say, each stage of a pipeline usually reads its input to completion, rather than reading a line at a time. This relates to #34, I think; if we had this kind of 'lazy reading' functionality, you could implement Unix tail -f, for example.

I can see what's needed, but I freely admit that right now I don't know how to do it. This needs some thought. I'd welcome suggestions!

[xxxserxxx]

@bitfield

package main

import (
	"bufio"
	"fmt"
	"os/exec"
	"time"
)

func main() {
	cmd := exec.Command("cat")
	inpipe, err := cmd.StdinPipe()
	if err != nil {
		panic(err)
	}
	go func() {
		for ittr := 0; ittr < 5; ittr++ {
			fmt.Fprintf(inpipe, "line number %d\n", ittr)
			time.Sleep(time.Second)
		}
		inpipe.Close()
	}()
	outpipe, err := cmd.StdoutPipe()
	if err != nil {
		panic(err)
	}
	err = cmd.Start()
	if err != nil {
		panic(err)
	}
	sc := bufio.NewScanner(outpipe)
	for sc.Scan() {
		fmt.Printf("%s: %s\n", time.Now().Format("15:04:05"), sc.Text())
	}
	cmd.Wait()
}

output:

$ go run ./iotest.go
08:59:33: line number 0
08:59:34: line number 1
08:59:35: line number 2
08:59:36: line number 3
08:59:37: line number 4

Is there a complexity that I don't see, or a limitation or downside to this approach?

Edit:

I forked the project and looked into the code. The architecture around Pipes makes such a solution less straightforward, which explains #34, and means that to submit a PR I'll have to understand about the model before suggesting a change. Before I go off and do that, do you have an approach or design you've been considering?

[bitfield]

Yes, using a bufio.Scanner sounds like a good approach!

Here's something you might like to try. Write a test for the behaviour you want (which will fail right now). For example, the test could (somehow) write one line to a pipe, and check that it can read that line from the other end of the pipe, even though the input reader hasn't yet been closed.

You can then use this test to try out ways of implementing this behaviour and seeing if they work.