Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Would toml be supported? #1364

Open
joshcangit opened this issue Sep 25, 2022 · 14 comments
Open

Would toml be supported? #1364

joshcangit opened this issue Sep 25, 2022 · 14 comments

Comments

@joshcangit
Copy link

joshcangit commented Sep 25, 2022

Please describe your feature request.

I wish I could use yq to work with toml files.

Note:

  • how to questions should be posted in the discussion board and not raised as an issue.
  • V3 will no longer have any enhancements.

Describe the solution you'd like

If we have example.toml like:

country = "Australia"

And we run a command:

yq -i toml '.country' example.toml

it could output

"Australia"

Describe alternatives you've considered

A clear and concise description of any alternative solutions or features you've considered.

I've tried another CLI to do this.
https://github.com/rossmacarthur/aq
But that one could only get input from stdin.

cat example.toml | aq -i toml '.country'

Additional context

There's another project of the same name.
https://github.com/kislyuk/yq

mikefarah/yq kislyuk/yq
Language Go Python
Dependencies From go.mod jq and from setup.py
json yq . input.json yq . input.json
yaml yq . input.yml yq -Y . input.yml
xml yq -p=xml -o=xml . input.xml xq -x . input.xml
toml tomlq -t . input.toml
csv yq -o=csv input.csv
tsv yq -o=tsv input.tsv
properties yq -p=props -o=props input.properties

Neither seem able to query csv, tsv or properties.

@mjperrone
Copy link

+1, I thought I had tomlq installed because I read the docs there (https://github.com/kislyuk/yq#toml-support) but I have this one so I am lacking the functionality. I'd love if this were supported here as well

@mikefarah
Copy link
Owner

Okay cool - I'll look into a toml go library 👍🏼

@mikefarah
Copy link
Owner

Hmm the problem I'm having with the available toml libraries in go is that they are lossy. They will not only drop comments, but also the order of the keys in maps :(

@mofosyne
Copy link

mofosyne commented Feb 5, 2023

Adding support to this ticket, another use case I've been hoping for a toml editor is for easier editing of /etc/samba/smb.conf in my automated raspberry pi print server configuration script I was writing.

https://gist.github.com/mofosyne/6baab7509ccd93f74d3fa225ea57d75d

In the absence of a suitable tool, I ended up using the diff and patch command, but that will likely break if there is a change to the smb.conf file in the future (but shall do for now). This feature would allow such script to be a little bit more robust.

@zamentur
Copy link

@mofosyne alternatively for this use case you can reuse ynh_write_var_in_file bash function from yunohost (AGPLv3):
https://github.com/YunoHost/yunohost/blob/dev/helpers/utils#L523
https://github.com/YunoHost/yunohost/blob/dev/helpers/utils#L600

it allows to do a lot of simple edition in a lot of different file format.

@bitdivine
Copy link
Sponsor

bitdivine commented Feb 23, 2023

@mikefarah It sounds as if the blocker is still the lossy go library. Is it worth someone making a non-lossy library or modifying an existing library to make it non-lossy? Did you find a library that you found particularly promising that you would prefer to use, if only it were not lossy?

If we use an existing library and improve it, we could start with lossy functionality and gradually improve until toml it is on a par with yaml in terms of functionality.

@bitdivine
Copy link
Sponsor

bitdivine commented Feb 23, 2023

That said, the premier python comment-preserving toml parser is 1000 lines. Might not be that hard to convert straight to go. We could keep that option open as well. I haven't written go in a while and am rather overloaded at the moment but I could have a shot when I have time (children and startups permitting and currently on 2 hours of sleep but one can dream of a time with time). This would help me so would also save time.
https://github.com/uiri/toml/blob/3f637dba5f68db63d4b30967fedda51c82459471/toml/decoder.py

@mikefarah
Copy link
Owner

I've made some progress with @pelletier toml library here: #1439 - I haven't looked at it since last year. From memory I think the parsing is now working reasonably well - but it will need some work to output back and remember which bits are array tables and so forth. @pelletier and I are both keen to get it working - if someone wanted to help that'd be appreciated...

@bitdivine
Copy link
Sponsor

I installed go and reminded myself about how it works, then cloned your branch, had a read through CONTRIBUTING.md and played with your code. I don't understand everything in the PR yet but I do note that even as it is, it is already useful. E.g. one of my original use cases was extracting some keys from a toml file to warm corresponding cache entries. The binary built in that branch already works perfectly for that use case.

@bitdivine
Copy link
Sponsor

Do you have specific examples of things that currently don't work, that are needed for, say, initial read-only support?

@mikefarah
Copy link
Owner

I'm planning on releasing a readonly version next release :)

@mikefarah
Copy link
Owner

4.33.1 has support for reading TOML!

@tspearconquest
Copy link

Wow, I was just looking earlier this week for a toml cli that worked like yq. Thank you, this is going to be very helpful!

@mcandre
Copy link

mcandre commented Mar 26, 2023

Good progress so far :)

toml is not yet supported as an output format

Next, would love to see this added as a built-in capability as well.

alexghr added a commit to AztecProtocol/aztec-packages that referenced this issue Dec 15, 2023
This PR fixes an issue with how we mirror code in the monorepo to
aztecprotocol/aztec-nr.

A while back the root aztec library added a dependency on
noir-protocol-circuits. In the monorepo this is achieved by use of a
relative path in Nargo.toml. This works fine for contracts that pull in
Aztec.nr through the monorepo (see also #3604) but it breaks projects
that use
[AztecProtocol/aztec-nr](https://github.com/AztecProtocol/aztec-nr)
because the relative path won't exist in the mirrored repo.

What this PR does is map any relative dependencies to protocol circuits
to a git dependency before pushing that commit to the mirrored repo. It
then undoes this change before pushing to the monorepo (we want to keep
using relative paths in the monorepo).

I would've preferred using `yq` since it claims it suports TOML (and is
included in the default Github actions package list) but its support is
limited to only basic types (so no objects like `{path="..."}`) and it
can only read toml but not write it.

mikefarah/yq#1364 (comment)
michaelelliot pushed a commit to Swoir/noir_rs that referenced this issue Feb 28, 2024
This PR fixes an issue with how we mirror code in the monorepo to
aztecprotocol/aztec-nr.

A while back the root aztec library added a dependency on
noir-protocol-circuits. In the monorepo this is achieved by use of a
relative path in Nargo.toml. This works fine for contracts that pull in
Aztec.nr through the monorepo (see also AztecProtocol#3604) but it breaks projects
that use
[AztecProtocol/aztec-nr](https://github.com/AztecProtocol/aztec-nr)
because the relative path won't exist in the mirrored repo.

What this PR does is map any relative dependencies to protocol circuits
to a git dependency before pushing that commit to the mirrored repo. It
then undoes this change before pushing to the monorepo (we want to keep
using relative paths in the monorepo).

I would've preferred using `yq` since it claims it suports TOML (and is
included in the default Github actions package list) but its support is
limited to only basic types (so no objects like `{path="..."}`) and it
can only read toml but not write it.

mikefarah/yq#1364 (comment)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

8 participants