xmlstar-devel Mailing List for XMLStarlet command line XML toolkit

Hi everyone,

perhaps the other heavy xmlstarlet users are also interested
in this, so I decided to share.

I wrote an LD_PRELOADable library (sorry, Macintosh users) that
can be used to make xmlstarlet fo/sel output a space before the
“/>” as is needed for XHTML.

https://mbsd.evolvis.org/cvs.cgi/contrib/hosted/tg/forceXHTML.c?rev=HEAD

This was not entirely simple as many useful functions for minimal
change are not exposed by the API; “fo” is a tad more fragile than
“sel” but I tested both on a small corpus and diff’d against the
normal results, but I don’t have PIs or anything else fancy in there.

I tested these on Debian GNU/Linux 11 (“bullseye”). These might even
work with other tools (xmllint/xsltproc) but I don’t use them, only
xmlstarlet, have “always” been.

Maybe drop me a note if you enjoy it, have patches or suggestions.

bye,
//mirabilos
-- 
„Cool, /usr/share/doc/mksh/examples/uhr.gz ist ja ein Grund,
mksh auf jedem System zu installieren.“
	-- XTaran auf der OpenRheinRuhr, ganz begeistert
(EN: “[…]uhr.gz is a reason to install mksh on every system.”)

On Wed, Aug 10, 2016 at 9:36 AM, Gary Madarm <gm...@gm...> wrote:
> I am trying to parse an XML file (https://static.nvd.nist.gov/
> feeds/xml/cve/nvdcve-2.0-2013.xml.gz) and access specific elements
> (vuln:cve-id, cvss:score and cpe-lang:fact-ref). I've tried many different
> variations of the command below but the result is always empty. How can I
> access these values?

There is a default namespace, see
http://xmlstar.sourceforge.net/doc/UG/ch05.html

Assuming a recent version of xmlstarlet (1.5+), you get the namespace
definitions for free:

xmlstarlet sel -t -v /_:nvd/_:entry/vuln:cve-id nvdcve-2.0-2013.xml

I am trying to parse an XML file (https://static.nvd.nist.gov/
feeds/xml/cve/nvdcve-2.0-2013.xml.gz) and access specific elements
(vuln:cve-id, cvss:score and cpe-lang:fact-ref). I've tried many different
variations of the command below but the result is always empty. How can I
access these values?

$ xmlstarlet sel -N scap-core="http://scap.nist.gov/schema/scap-core/0.1"
-N cvss="http://scap.nist.gov/schema/cvss-v2/0.2" -N vuln="
http://scap.nist.gov/schema/vulnerability/0.4" -N xsi="
http://www.w3.org/2001/XMLSchema-instance" -N patch="
http://scap.nist.gov/schema/patch/0.1" -N cpe-lang="
http://cpe.mitre.org/language/2.0" -t -v ''/nvd/entry/vuln:cve-id"
nvdcve-2.0-2013.xml
$

FYI, <item key="legendTemplate"><![CDATA</item> is not valid XML,
that's why xmlstarlet ed will never generate it.

Hi all,
I meanwhile solved it myself. I have to run the result trough xmlstarlet
unesc

Issue solved.

Mit freundlichen Grüßen / Kind Regards
Hr./Mr. Karl-Heinz Fischbach

-----Ursprüngliche Nachricht-----
Von: Averlon GmbH (in Gründung) [mailto:in...@av...] 
Gesendet: Mittwoch, 20. April 2016 11:01
An: xml...@li...
Betreff: [Xmlstar-devel] xmlstarlet creates curious values

Hi all,

 

I have tried to set a value for a xml attribute like this:

 

xmlstarlet ed --update
"/charts/lineChart/data[@id='lineChartOptions']/item[@key='legendTemplate']"
-v '<![CDATA' /tmp/input.av_chart.xml

 

The value is nonsense but for testing purposes.

 

Result is:

 

<item key="legendTemplate">&lt;![CDATA</item>

 

I also tried the global parameter “—pf” but same result.

 

What can I do different to preserve the value as written in the command?

 

Mit freundlichen Grüßen / Kind Regards

Hr./Mr. Karl-Heinz Fischbach



 

----------------------------------------------------------------------------
--
Find and fix application performance issues faster with Applications Manager
Applications Manager provides deep performance insights into multiple tiers
of your business applications. It resolves application problems quickly and
reduces your MTTR. Get your free trial!
https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
_______________________________________________
xmlstar-devel mailing list
xml...@li...
https://lists.sourceforge.net/lists/listinfo/xmlstar-devel

Hi all,

 

I have tried to set a value for a xml attribute like this:

 

xmlstarlet ed --update
"/charts/lineChart/data[@id='lineChartOptions']/item[@key='legendTemplate']"
-v '<![CDATA' /tmp/input.av_chart.xml

 

The value is nonsense but for testing purposes.

 

Result is:

 

<item key="legendTemplate">&lt;![CDATA</item>

 

I also tried the global parameter “—pf” but same result.

 

What can I do different to preserve the value as written in the command?

 

Mit freundlichen Grüßen / Kind Regards

Hr./Mr. Karl-Heinz Fischbach



 

On Sat, May 16, 2015 at 11:49 AM, Thomas Petazzoni
<tho...@fr...> wrote:
> When libgcrypt is used, it is linked with libgpg-error, so we should
> also test that libgpg-error is available, and include -lgpg-error in
> the LIBS variable.

Thanks, applied.

https://sourceforge.net/p/xmlstar/code/ci/0380e4d1e2d02aec868b7199e71bbe56e8161223/

>
> Note that using PKG_CHECK_MODULES() would be a much much better
> replacement than this complicated handling of static libraries, but
> it's a much more significant effort.

I don't think PKG_CHECK_MODULES() handles static libraries. Unless you
did ./configure --enable-static-libs you will be using shared
libraries anyway which don't need any of these checks.

When libgcrypt is used, it is linked with libgpg-error, so we should
also test that libgpg-error is available, and include -lgpg-error in
the LIBS variable.

This fixes build issues like:

  CCLD     xml
/home/thomas/projets/buildroot/output/host/usr/bfin-buildroot-uclinux-uclibc/sysroot/usr/lib/libexslt.a(crypto.o): In function `_exsltCryptoGcryptInit':
crypto.c:(.text+0x112): undefined reference to `_gcry_check_version'
/home/thomas/projets/buildroot/output/host/usr/bfin-buildroot-uclinux-uclibc/sysroot/usr/lib/libexslt.a(crypto.o): In function `_exsltCryptoRc4DecryptFunction':
crypto.c:(.text+0x316): undefined reference to `_gcry_cipher_open'
crypto.c:(.text+0x32a): undefined reference to `_gcry_strerror'
crypto.c:(.text+0x34c): undefined reference to `_gcry_cipher_setkey'

Which are caused by the AC_SEARCH_LIBS() test for libgcrypt to fail
due to -lgpg-error not been present in the LIBS variable.

Note that using PKG_CHECK_MODULES() would be a much much better
replacement than this complicated handling of static libraries, but
it's a much more significant effort.

Signed-off-by: Thomas Petazzoni <tho...@fr...>
---
 configure.ac | 1 +
 1 file changed, 1 insertion(+)

diff --git a/configure.ac b/configure.ac
index 77dcc23..63a26ce 100644
--- a/configure.ac
+++ b/configure.ac
@@ -127,6 +127,7 @@ AS_IF([test "x$STATIC_LIBS" != xno],
     [AC_SEARCH_LIBS([libiconv_open], [iconv], [], [], "$USER_LIBS")], "$USER_LIBS")
     AC_SEARCH_LIBS([clock_gettime], [rt], [], [], "$USER_LIBS")
     AC_SEARCH_LIBS([deflate], [z], [], [], "$USER_LIBS")
+    AC_SEARCH_LIBS([gpg_strerror], [gpg-error], [], [], "$USER_LIBS")
     AC_SEARCH_LIBS([gcry_cipher_encrypt], [gcrypt], [], [], "$USER_LIBS")
 
     # Checks for inet libraries:
-- 
2.1.0

On Tue, Oct 7, 2014 at 6:21 PM, Jamie Jackson <jam...@gm...> wrote:
> Can XMLStarlet update XML with sorted elements? (Or is some other utility
> more appropriate?)

XSLT can sort elements: http://www.w3.org/TR/xslt#sorting

XMLStarlet won't help you much here because the ed(it) command isn't
based on XSLT, and it isn't really practical to edit files with the
sel(ect) command. You can use the tr(ansform) command to apply XSLT,
though that isn't different from using xsltproc.

Hi Folks,

In version control, I am storing some XML configuration documents that are
machine-generated. The machine sometimes arbitrarily changes the order of
some elements, and that leads to spurious diffs. I'd like to scrub those
files as part of my config downloading routines by introducing certain
sorts. (I don't much care what they are, as long as they're consistent.
Ascending alphabetic sort, based on some specified XPath level would be
fine.)

Can XMLStarlet update XML with sorted elements? (Or is some other utility
more appropriate?)

If it's easy, I'll probably implement it; otherwise, I'll probably learn to
live with the spurious diffs.

Thanks,
Jamie

On Sun, Mar 30, 2014 at 3:48 AM,  <kir...@us...> wrote:
> Sorry to trouble you, but I can not find the document which describes
> functions available in xmlstarlet.

Please send queries regarding xmlstarlet to the mailing list, or the
project forums.

By functions I assume you mean XPath ones. These are all from libxml
and libxslt, they are documented at
http://www.w3.org/TR/xpath/#corelib and
http://www.w3.org/TR/xslt#add-func. Also, many of the functions from
http://exslt.org/.

On Fri, Nov 29, 2013 at 12:26 AM,  <ans...@us...> wrote:
> I met a problem when using ed, like this
> xml ed -u "/ns2:testCase/scenario/description" -v "haha lala /& gt;"
> test_case_01_normal22.xml
> But the result is
> "\<description>haha lala /& amp;gt;\</description>"

I don't understand where the backslashes and space after the "&" are
coming from, I tried something similar and got

<xml>haha lala /&amp; gt;</xml>

> How to make & just print &?

That would be invalid XML, xmlstarlet will not produce it. In XML,
"&amp;" means "&".

PS: please send xmlstarlet questions to the mailing list (I've added
it to cc), or make a support request at
http://sourceforge.net/p/xmlstar/support-requests/.

I have some scripts to ./configure xmlstarlet and dependant libs, I
just put them into git in the mingw/ directory (should have done that
a while ago).

http://sourceforge.net/p/xmlstar/code/ci/master/tree/mingw/

Filipe Fernandes <ffe...@gm...> writes:

> Thanks!  That definitely got me further along.  The manpage was built,
> but I still had to contend with the following error...
>
>   DBOOK doc/xmlstarlet.1
> Note: Writing xmlstarlet.1
> make[1]: *** No rule to make target `doc/xmlstarlet-ug.pdf', needed by
> `all-am'.  Stop.
> make[1]: Leaving directory `/home/ffernand/src/.interest/xmlstar'
> make: *** [all] Error 2

I guess you ignored the warning about not having fop installed. Maybe it
should be an error since it breaks the build? On the other hand, the
tarball has the pdfs so it wouldn't break the build there...

On Thu, Jan 24, 2013 at 2:51 PM, Noam Postavsky
<npo...@us...> wrote:
>
> On Thu, Jan 24, 2013 at 2:36 PM, Filipe Fernandes
> <ffe...@gm...> wrote:
> > It got me further along... but there appears to be another hiccup...
> >
> > $ make
> > make  all-am
>
> >   CCLD   xml
> > make[1]: *** No rule to make target `doc/xmlstarlet.1', needed by `all-am'.
>
> > I'm taking a look myself as I try to trace through the all-am target... but
> > I thought I'd mention it in-case it was obvious to you.
>
> Hmm, I think you need to pass --enable-build-docs to ./configure. I
> had the idea that documentation building rules should be disabled by
> default because they require installing Apache FOP (also it runs
> pretty slow). The tarball comes with the docs prebuilt, but the
> default build from git is broken as you have just discovered; not sure
> what the best fix for that is.

Thanks!  That definitely got me further along.  The manpage was built,
but I still had to contend with the following error...

  DBOOK doc/xmlstarlet.1
Note: Writing xmlstarlet.1
make[1]: *** No rule to make target `doc/xmlstarlet-ug.pdf', needed by
`all-am'.  Stop.
make[1]: Leaving directory `/home/ffernand/src/.interest/xmlstar'
make: *** [all] Error 2

I don't need the pdf, so in the auto-generated Makefile I changed the line...

    dist_doc_DATA = $(userguide_gen) doc/html.css $(txtguide)

by removing $(userguide_gen) in order to continue with the `make install`.

I was able to build and install (including manpages).  Thanks for
that!  I know that doesn't help you much :(

much appreciated,
filipe

On Thu, Jan 24, 2013 at 2:36 PM, Filipe Fernandes
<ffe...@gm...> wrote:
> It got me further along... but there appears to be another hiccup...
>
> $ make
> make  all-am

>   CCLD   xml
> make[1]: *** No rule to make target `doc/xmlstarlet.1', needed by `all-am'.

> I'm taking a look myself as I try to trace through the all-am target... but
> I thought I'd mention it in-case it was obvious to you.

Hmm, I think you need to pass --enable-build-docs to ./configure. I
had the idea that documentation building rules should be disabled by
default because they require installing Apache FOP (also it runs
pretty slow). The tarball comes with the docs prebuilt, but the
default build from git is broken as you have just discovered; not sure
what the best fix for that is.

On Wed, Jan 23, 2013 at 9:31 PM, Noam Postavsky <
npo...@us...> wrote:

>
> mawk wants a different number of backslashes than gawk to output a
> backslash in gsub(). After taking a short look at [1] I decided I really
> don't want to know why, so I've rewritten usage2c.awk to avoid
> backslashes in gsub(). Please try the latest git (e9cd8843cb1f27).
>
> [1] http://www.gnu.org/software/gawk/manual/html_node/Gory-Details.html
>

Thanks Noam for the quick reply,

It got me further along... but there appears to be another hiccup...

$ make
make  all-am
make[1]: Entering directory `/home/ffernand/src/.interest/xmlstar'
  CC     src/trans.o
version.h: #define VERSION "1.4.2-2-ge9cd884"
  CC     src/xml.o
  CC     src/xml_C14N.o
  CC     src/xml_depyx.o
  CC     src/xml_edit.o
  CC     src/xml_elem.o
  CC     src/xml_escape.o
  CC     src/xml_format.o
  CC     src/xml_ls.o
src/xml_ls.c: In function ‘xml_print_dir’:
src/xml_ls.c:140:9: warning: ignoring return value of ‘chdir’, declared
with attribute warn_unused_result [-Wunused-result]
  CC     src/xml_pyx.o
  CC     src/xml_select.o
  CC     src/xml_trans.o
  CC     src/xml_validate.o
  GEN    src/usage.c
  CC     src/usage.o
  GEN    src/c14n-usage.c
  CC     src/c14n-usage.o
  GEN    src/depyx-usage.c
  CC     src/depyx-usage.o
  GEN    src/edit-usage.c
  CC     src/edit-usage.o
  GEN    src/elem-usage.c
  CC     src/elem-usage.o
  GEN    src/escape-usage.c
  CC     src/escape-usage.o
  GEN    src/format-usage.c
  CC     src/format-usage.o
  GEN    src/ls-usage.c
  CC     src/ls-usage.o
  GEN    src/pyx-usage.c
  CC     src/pyx-usage.o
  GEN    src/select-usage.c
  CC     src/select-usage.o
  GEN    src/trans-usage.c
  CC     src/trans-usage.o
  GEN    src/unescape-usage.c
  CC     src/unescape-usage.o
  GEN    src/validate-usage.c
  CC     src/validate-usage.o
  CCLD   xml
make[1]: *** No rule to make target `doc/xmlstarlet.1', needed by `all-am'.
 Stop.
make[1]: Leaving directory `/home/ffernand/src/.interest/xmlstar'
make: *** [all] Error 2

I'm taking a look myself as I try to trace through the all-am target... but
I thought I'd mention it in-case it was obvious to you.

thanks,
filipe

Filipe Fernandes <ffe...@gm...> writes:
>
> I looked at line 55 of src/select-usage.c and it does contain the character
> '\', which I believe in C should be '\\'.  src/select-usage.c appears to be
> auto-generated using the script usage2c.awk script.

mawk wants a different number of backslashes than gawk to output a
backslash in gsub(). After taking a short look at [1] I decided I really
don't want to know why, so I've rewritten usage2c.awk to avoid
backslashes in gsub(). Please try the latest git (e9cd8843cb1f27).

[1] http://www.gnu.org/software/gawk/manual/html_node/Gory-Details.html

On Wed, Jan 23, 2013 at 11:50 AM, Filipe Fernandes
<ffe...@gm...>wrote:

> Hello,
>
> I've been able to reliably build 1.3.x versions from the git repository in
> the past, but I can no longer build 1.4.x when I checkout git tags.
>

Oh... forgot to mention that I'm using Ubuntu 12.04 (64-bit).

filipe

Hello,

I've been able to reliably build 1.3.x versions from the git repository in
the past, but I can no longer build 1.4.x when I checkout git tags.

I go through the normal routine..

$ autoreconf -sif
$ ./configure
$ make

At the `make` command it complains of the following...

$ make
make  all-am
make[1]: Entering directory `/home/ffernand/src/.interest/xmlstar'
  CC     src/trans.o
version.h: #define VERSION "1.4.2"
  CC     src/xml.o
  CC     src/xml_C14N.o
  CC     src/xml_depyx.o
  CC     src/xml_edit.o
  CC     src/xml_elem.o
  CC     src/xml_escape.o
  CC     src/xml_format.o
  CC     src/xml_ls.o
src/xml_ls.c: In function ‘xml_print_dir’:
src/xml_ls.c:140:9: warning: ignoring return value of ‘chdir’, declared
with attribute warn_unused_result [-Wunused-result]
  CC     src/xml_pyx.o
  CC     src/xml_select.o
  CC     src/xml_trans.o
  CC     src/xml_validate.o
  GEN    src/usage.c
  CC     src/usage.o
  GEN    src/c14n-usage.c
  CC     src/c14n-usage.o
  GEN    src/depyx-usage.c
  CC     src/depyx-usage.o
  GEN    src/edit-usage.c
  CC     src/edit-usage.o
  GEN    src/elem-usage.c
  CC     src/elem-usage.o
  GEN    src/escape-usage.c
  CC     src/escape-usage.o
  GEN    src/format-usage.c
  CC     src/format-usage.o
  GEN    src/ls-usage.c
  CC     src/ls-usage.o
  GEN    src/pyx-usage.c
  CC     src/pyx-usage.o
  GEN    src/select-usage.c
  CC     src/select-usage.o
src/select-usage.c:55:233: warning: multi-character character constant
[-Wmultichar]
src/select-usage.c:55:1: error: stray ‘\’ in program
src/select-usage.c:55:1: warning: overflow in implicit constant conversion
[-Woverflow]
src/select-usage.c:55:239: error: expected ‘}’ before ‘n’
src/select-usage.c:55:240: warning: missing terminating ' character
[enabled by default]
src/select-usage.c:55:239: error: missing terminating ' character
make[1]: *** [src/select-usage.o] Error 1
make[1]: Leaving directory `/home/ffernand/src/.interest/xmlstar'
make: *** [all] Error 2


I looked at line 55 of src/select-usage.c and it does contain the character
'\', which I believe in C should be '\\'.  src/select-usage.c appears to be
auto-generated using the script usage2c.awk script.

filipe

Kai Hendry <he...@ik...> writes:
> curl -s -G --data-urlencode r=XML --data-urlencode i=tt0988595
> http://www.omdbapi.com/ | xmlstarlet pyx
>
>
> I think it might be this issue:
>
http://sourceforge.net/tracker/?func=detail&aid=3595212&group_id=66612&atid=515106

Yup, I was a bit lazy about releasing the fix, but you can get it from
1.4.2 now.

1.4.2: Dec 28, 2012

    - pyx: avoid segfault on documents with multiple attributes (Bug
      #3595212)

1.4.1: Dec 8, 2012

    - avoid segfault when attempting to edit the document node (Bug
      #3575722)

    - Packaging:
      - include doc/xmlstar-fodoc-style.xsl in the dist so that the
        --enable-build-docs option works from the tarball (Bug
        #3580667)
      - AC_SUBST PACKAGE_TARNAME for automake so that documentation is
        installed to the right place (Bug #3561958)

    - Test Suite:
      - avoid test failures due to XML formatting and whitespace
        changes (also fixes Bug #3572789)
      - use automake's parallel test suite
      - make bigxml tests much faster by using whitespace instead of nodes
      - don't test str:replace() with ed: it doesn't work outside of
        xslt in new libxslt
      - ignore extra errors from libxml 2.9.0 bug
      - let tests run using busybox
      - add runAllTests.sh to run tests without make

1.4.0: Aug 13, 2012

    - Documentation:
        - executable name used in documentation now matches
          --transform-program-name (Bug #3283713)
        - added Makefile rules for generating documentation
          (./configure --enable-build-docs)

    - ed subcommand:
        - relative XPaths are now handled correctly (Bug #3527850)
        - the last nodeset inserted by an edit operation can be
          accessed as the XPath variable $prev (or $xstar:prev)
        - add --var option to define XPath variables
        - allow ed -u -x to insert nodesets instead of converting to
          string
        - remove hard limit for number of edit operations (Bug
          #3488240)

    - pyx now handles namespaces correctly

Алексей Штыков <if...@ma...> writes:

> Hello,
> I have planned to wait until this friday, but I think this guy have found another solution.
> I thought to made additional command for comments, but extending an "ed" command looks better.
> I will write an answer under the request and try to write some code at this weekend.

Did you have a chance to look at this?