We cracked the 70-stars mark on github! Wow. We reached the 2,000-commits-mark. Wow. We also reached the 500 PR/Issue mark. Another wow.

But there happened a lot more things in the last 14 days in imag, the personal information management suite for the commandline. Read it up here.

The past

Wow. 14 days can be so damn short. We got a few things done in the last 14 days, but first, I want to thank all contributors:

• mario-kr who did a semester project at his university with some fellow students. He submitted a pull request to implement libimagtodo, which is a todo module for imag, currently using task-hookrs as backend for talking to taskwarrior. This is really great!
• mario-kr also submitted some more PRs:
  • A bugfix for the bin/imag binary
  • A feature for task-hookrs to import a task from taskwarrior from a single JSON object.
  • A lot of suggestions and comments in issues what is wrong with task-hookrs.
• jsirois opened two PRs after reading my this-week-in-rust entries where I submitted two issues which could be done by people wanting to learn some Rust: 531 and 532 – absolutely awesome!
• I also want to thank impo who submitted a PR to build the docs to a manpage. This reminds me that I really should work on the docs. Either way, a big “Thank you!” to you, impo!

Wow, so much contributions! Awesome, keep it up, guys! Lets take a look on the merged pull requests.

PRs merged

The following PRs were merged in the last 14 days:

• 468 added libimagref - a library to create references to files on the filesystem. Something like pointers to external data. This can be used to reference files outside of the store, track them by content (SHA1), their permissions, their location and so on. This will also be used to implement a list of other tools, such as a wiki module, a movie-, music-, images-module and so on. There are a lot of possibilities here.
• 481 implements a frontend for libimagref for the commandline, so a user can use libimagref directly to create filesystem references.
• 487 added an implementation of Display for StoreId. You can now println!("{}", id) – awesome!
• 489 refactored the internals of the store a bit, by moving a function from Store to StoreId. This shouldn't change the use of libimagstore at all.
• 496 made the CoreLister-Type from libimagentrylist more generic. You should be able to pass not only closures, but also normal functions now.
• 500 contained a bug-fix for 468
• 503 fixed a flaw in the Display implementation for StoreId
• 513 adds a configuration option which makes it possible to deny altering hooks per aspect in the store hook execution
• 517 fixed a bug in libimaginteraction which caused the ask_bool() (and possibly other) function to loop endlessly
• 518 removes a unneeded dependency from imag-view
• 519 puts the out directory in the gitignore file
• 521 adds the implementation for libimagbookmark – the bookmark feature of imag (the library part).
• 525 updates the documentation. Before, the documentation was a technical document. I do not believe that this will bring us any advantages anymore. This PR rewrites the document to be a user documentation.
• 528 updates all the crate versions from “0.1.0” to “0.2.0”, as we currently develop imag in this newer version.
• 531 removes some unused code and therefor cleans up the warnings the compiler gives us about the unused code. This PR was later superceded by 536 and 537.
• 532 adds a fold_result utility function for all iterators and therefor closes
• 533 fixed a header-field naming error in libimagref.
• 534 introduces a (hopefully) temporary fix for libimagentrylink. The check, whether a StoreId object points to an external link store object or not was buggy as it assumed the PathBuf inside the StoreId to be absolute to the Store root.
• 536 superceded 531
• 537 superceded 531
• 538 refactored some code in the store to remove duplication.

PRs about to be merged

And here are some PRs which are work in progress and might be merged in the next 14 days. These were opened in the last 14 days and I hope to close them (merged or not merged) fast... some of them seem to be pointless (the ones on the store refactoring for testing are kind of duplicated as I tried different approaches) so they might get closed unmerged.

• 486 implements a git store hook, which then puts the store into a git repository and tracks changes within the store. I'm not sure whether I will be able to merge this, as I still have problems getting the git2 crate building, as zlib is somehow missing, though I added it to my nixos setup... we will see...
• 497 implements an iterator type for libimagref, so we can iterate over references in a nice way.
• 498 uses 497 in imag-ref (481) to list references if the user requests this.
• 514 adds an abstraction layer for the filesystem which should simplify store testing option which makes it possible to deny altering hooks per aspect in the store hook execution
• 522 adds the implementation for imag-bookmark – the frontend for libimagbookmark, which is implemented in #521 - so this is basically the other half of the feature.
• 529 is an attempt to pass only store-absolute StoreId-instances to the Entry objects inside the store. This PR might be superceded by 530
• 530 tries to rewrite the Store internals to use another StoreId type (EntryStoreId) inside the Store implementation which is always absolute to the Store root and never relative to the filesystem root. 499, which is absolutely awesome.
• 535 introduces a feature for imag-counter, which gives you the possibility to add units to your counters (when creating them). So you can attach a meaning to the counters and everybody will know that this is not the count of your daily pushups but rather the number of jellybaby you've eaten!

The Future

So what will happen next? I opened quite a bunch of issues in the last 14 days, most of them are rather simple things to do!

Issues opened/I want to get done

We reached the 60-open-issues line as well in the last 14 days. This is not necessarily a bad thing, as these are not all bugs but also feature requests and so on. Here is a list of issues which were opened in the last two weeks. I hope I can close some of them within a few days (some of them might be already closed at the time of writing).

• 488 [open]
• 491 [closed]
• 493 [open]
• 494 [open]
• 495 [open]
• 499 [open]
• 501 [open]
• 505 [open]
• 506 [open]
• 507 [open]
• 508 [open]
• 509 [closed]
• 511 [open]
• 515 [closed]
• 516 [open]
• 520 [open]
• 523 [open]
• 524 [open]
• 526 [closed]
• 527 [open]

I'm not including a description of each issue here... this would yield this post waaaay to long actually.

Other things

Well, I am in the last phase of my bachelors thesis right now... so I would say: Plenty of time to work on imag, right?

I hope to get the bookmark implementation finished and the listing support for the reference utility merged. Besides that (and after the reference foo is merged) I would love to start implementing a backend for the diary or bookmark module by either using jrnl.sh or buku. Not because I use them as commandline tools (I'm not saying I wouldn't) but because they seem to be simple to adapt. Maybe I'm wrong... but well... lets see.

tags: #linux #open source #programming #rust #software #tools #imag

I just released the second version of my task-hookrs crate, a crate for interfacing with taskwarrior!

This release is release 0.2.0 – so nothing too serious by now, as we still prefix with 0., but anyways a few nice things were merged and it seems to work pretty well!

These features were added:

• Annotation support
• Optional values are no longer included in the output
• import_task() for importing a single task JSON object

Bugfixes and other things were also done:

• Dead code removed
• Documentation written and updated
• Links in documentation fixed
• Import errors for rust beta fixed

And the crate is already published on crates.io.

So go ahead and use it!

tags: #linux #open source #programming #rust #software #tools

imag has made little progress in the last 14 days, as I'm really focusing on my bachelors thesis and exams right now. Anyways, I got some neat things done. Read here what happened in the last 14 days.

The past

Yes, my thesis is my main focus right now. This does not mean that imag is stalled, indeed I got four merges done in the last 14 days, and some more are waiting.

Developers Certificate of Origin

I added a Developers Certificate of Origin in a pull request. This means, that all developers/contributors of imag agree to the terms listed in this certificate.

Within this pull request I also added some notes how to contribute to imag without having a github account. People are welcome to send me patches for imag via mail! They just have to acknowledge that I will integrate them by opening a PR on github myself. Also, they have to add a "Signed-off-by:" tag in their commit messages, I will not include patches send by mail without this line.

I think that shouldn't be an issue.

Result::map_err_into()

This was the second pull request. It was followed by two pull requests which moved the codebase to use the feature from this PR.

What it implements: An additional method for the Result type, so one can use the Result::map_err_into() method now to minify error handling code in a sane way.

Before this pull request, one had to write the following code over and over again, to wrap an error type into the module-specific error type:

some_operation_that_might_fail()
    .map_err(Box::new)
    .map_err(|e| ModuleError::new(ModuleErrorKind::FailingOperation, Some(e)))

This quickly became rather ugly. With the new feature applied, we can write this now in a really short and neat way. The following is equal to the code snippet above:

some_operation_that_might_fail()
    .map_err_into(ModuleErrorKind::FailingOperation)

With the map_err_into() method doing the heavy lifting of packing the error into a new Box and creating an error instance with the passed error kind.

As this was implemented in the libimagerror code generator macro, this is available for all crates which use the libimagerror macro generate_error_types automatically.

The PRs that followed that improvement removed a lot of boilerplate code:

4 files changed, 63 insertions(+), 113 deletions(-)

The Future

Lets see what's in the pipeline. Most of the following things are “I want to do this” things, and I won't be able to do them that soon, as the exams are coming up. I really hope I can get some progress, still.

References

The libimagref library is in progress. This library will enable the user to reference files outside of the store. The imag-ref binary crate will be a frontend to use this library.

This is in progress and I'm really looking forward to it. I will use the libimagref library to write a wiki module with vimwiki as backend, after this gets merged.

store hooks: git

I want to continue to work on the git hook layer, so we finally have a version control system in the store.

Filter language

Another thing I really want to do is to implement the filter language frontend, so one can use libimagentryfilter to implement a filter right on the commandline.

The best option would be to add two/multiple frontends for this:

• hlua
• ketos (a lisp dialect)
• rhai

so one can select one of these languages by commandline and then filter entries.

That would be a really huge amount of work, but it would be fun, really.

LGPL checker

Another thing what I want to do is to add the LGPL header in each source file. The point is: I want to check this via the travis CI setup, so travis checks whether all files contain the LGPL header. That is possible, I did not yet figure out how to do it properly, though.

tags: #linux #open source #programming #rust #software #tools #imag

As imag gets more traction, I get asked more often “What is its purpose?” or “What does it do?” – here's an explanation!

After I posted my 8th iteration on “What's coming up in imag” on reddit I noticed a huge increase of the number of stars of the project. It went from less than 20 stars up to over 50 within one week. Amazing!

But I also got questions: What is this “imag” thing you are writing – what does it do? So here's an attempt to explain.

A CLI PIM suite?

The README file in the imag source repository states that imag is a personal information management suite for the commandline. Its goal is

Create a fast, reliable, forwards/backwards compatible commandline personal information management suite which covers all aspects of personal information management, consists of reusable parts and integrates well with known commandline tools.

Which is exactly what it is.

Yes, but...

imag is a CLI PIM suite but tries to implement as little functionality as possible – how does it do that? Well, we try to re-use existing tools. We do not implement the PIM tools ourselves, but we re-use known tools, so a user does not have to learn a new tool when switching to imag – she can simply add imag to her setup!

(Bad visualization skills ahead!)

As the picture above shows, all these fancy tools have some sort of database where they store their data. This is awesome and works – what it doesn't provide is to connect this data and add annotations to the connections – that's where imag steps in:

As the picture above shows, imag tracks the data other tools create (it does not duplicate it). It provides ways to link this data and therefor creating a network of data you can query later.

Ah, and what does work by now?

Not that much, as it is an really early stage right now. Some of the modules are already implemented, namingly the

• counter module
• notes module
• diary module

as well as some utility modules for more plumbing-like commands:

• imag-link for linking entries together
• imag-tag for adding, removing and altering tags of an entry
• imag-view for viewing entries. It only supports printing to commandline as of this writing, but will support more in the future, including showing dependency graphs of entries, compiling markdown and showing it in the browser and so on.
• imag-store A commandline-interface to the store itself. This shouldn't be used by the end-user and is for debugging purposes and development work. You can destroy data with this tool rather easily, so if you want to play around, make sure you have a backup of your data!

All tools mentioned above are implemented without using third-party software, which means that there are no tools other than imag which need to be installed to use the advertised functionality (of course imag itself depends on libraries). Future work will be based on other software, so the todo module will be based on taskwarrior, the ledger module will be based on beancount and so on.

The plan

Currently, I'm having in mind to add these tools to the imag suite:

• Todo module: taskwarrior
• Ledger module: beancount
• lend-tracker module: [no backend tool found yet] – I guess I will implement this myself as I cannot think of any complex functionality here.
• bibliography module: [no backend tool found yet] – I doubt that there are commandline tools available for organizing bibtex files. You are welcome to proof me wrong!
• bookmark module: [self-implementation] – I will implement bookmark functionality myself if I do not find a nice commandline tool which fits my needs.
• shopping list module: [not decided yet] – I want to create a module which helps me doing my shopping lists. I'm not sure whether there is a CLI tool for this available – I doubt it actually.
• calendar module: [self-implementation] – I will implement this functionality myself using the icalendar data format if I do not find some nice tool. Maybe khal is good enough, but I'm not sure it offers an API.
• contact module: [self-implementation] – Same as for the calendar module: I will implement this myself using the vcard data format if I cannot find a nice commandline tool. Maybe khard fits my needs, but I'm also pretty sure that this one does not offer an API either
• cue cards module: anki (not sure here, as anki is not a commandline tool or does not have a CLI interface (yet?))
• mail module: [self-implementation] – I'm pretty sure I have to implement this myself, because beeing able to reference emails means that I have to be able to read each of the mails anyways. Maybe this will be a complex module offering functionality to open mutt for mails/mailboxes and query sup or notmuch for more information about mails... - The possibilities are rather endless in this case and I think this will be the most complex tool in the imag suite.
• image module: [self-implementation] – The image module needs to be able to read the meta information of images and give me the possibility to query them for meta-information. I guess I can implement this myself.
• movie module: [self-implementation] – Here goes the same as for the image module – a module to query meta-information of movie files.
• music module: [self-implementation] – and another one equal to the image module – a module to query meta-information of music files.
• news module: newsbeuter – I'm a bit insecure because newsbeuter does not offer an API and uses a sqlite database as data storage. I would need to read this database and try to get my information from there. This is do-able in rust, but I'm not sure whether I like it or not. There should be a non-curses CLI interface for newsbeuter, really! This not only includes RSS and ATOM, but maybe also usenet and podcasts.
• password module: password-store – This is a home-run, as I won't do much besides creating pointers inside the store to each password file – so absolutely no complexity here!
• weather module: [not decided yet] – I'm not sure what commandline tools are out there that fit my needs here. For starters, I just want to be able to put weather information into the store and refer to it from other files. I've not read into the topic at all by now, so I won't promise things here.
• weight tracker module: [no backend tool found yet] – Are there commandline tools for weight tracking available? I would love to have one!
• workout module: [not decided yet] – This just came up while writing this blog post. Pretty much the same as for the weight tracker module... this could even include the weight tracker module!
• wiki module: [no backend tool found yet/not decided yet] – There are some wiki CLI tools available, but I'm not that happy with either of them. I would use vim-wiki personally, but it is rather hard to learn all the commands and I'm pretty sure I would screw up pretty bad when starting to use it actively. I wouldn't include vim-wiki in imag either, because I want to be able to be editor-agnostic.
• project management module: This is nothing more than a brain-fart by now, but maybe I can add a module to do issue tracking and project notes and so on. This shouldn't replace a bug tracker or something, just a personal-project-helper tool or something like that.

Infrastructure needed

I already wrote a lot of infrastructure libraries for imag, including but not limited to functionality for listing, viewing, tagging, linking filtering entries.

There are also libraries for user interaction, selecting entries on the commandline, markdown parsing or time specifying on the commandline to enhance the user experience.

Some more libraries for error handling, store hooks and runtime functionality are in the imag codebase for simplifying the process of writing imag modules.

And, of course, there is the very heart of imag, the store library. This is the library which ties everything together and the univeral interface to the filesystem for all the functionality in imag.

There is also a library in progress for referencing files which do not live in the imag store which can be used to “link” files into the imag store by creating pointers to them. It doesn't use symlinks, as symlinks can be killed by moving the original file. It also doesn't use hardlinks, as the user would have to remove two files instead of one when she removes data from her system – and this is clearly not feasible. So this library simply implements some kind of pointer to files. It also hashes the content of files to be able to re-find a file if it is moved (as long as the content does not change, of course).

The most important part which is almost not covered yet is testing. There is so much code to test.

My wishes for the future.

I have a varity of hopes and wishes for the future. The most important one is

Contributors!

I really really hope for contributors. I mean, I really hope some people start contributing. imag has grown to be my never-ending-project, as there are so many things to do, but I also think such a tool was not created before – so there is a lot of potential to discover and things to discuss or try out!

I think, though, I have layed out the groundwork on this. I created a list of issues in the github repository which are marked with the complexity/easy label so there's a list of easy-to-do issues which a contributor could check and try out.

I also write a two-weekly blog series on what is going on in the project (1, 2, 3, 4, 5, 6, 7, 8, 9) – I also post these on reddit in the /r/rust subreddit – so I really try to spread the word.

Flexible backends

This is another big thing. I wrote down a list of tools I want to integrate into imag above. All functionalities should re-use existing tools to cover there needs, I do not want to reimplement things others have implement very well before.

But, maybe someone does not like my choises and wants to use some other tool than I have used to integrate into imag. Or the maintainer of the external tool stops working on the tool – what then?

I want to create an abstraction layer over the backends, so one can simply switch from (for example) taskwarrior to imaginary-other-todo-tool.

That would be a really complex library, as it has to be completely tool and use-case agnostic. But I think it is do-able, just not yet, as we do not have the experience of binding several tools to imag.

Moving away from github (actually not...)

I'm also tempted to move away from github. The only thing that holds me there is the urge to find contributors and github is the easiest way to do so in my opinion. Also, if I move away from github, I really want to use text-only tools for developing imag – just as in the spirit of imag itself. The problem is, that there is no plain-text bug tracking software available that fits my needs (I wrote about that before). I could use a mailing list and ask contributors to send patches by mail. But I don't like that approach that much.

Also, some of the conveniences that github offers are rather nice. If I would do the move, I would setup a website, CI system, issue tracker, git repositories including a web-frontend (gitweb or similar, not a full-blown gogs or something), wiki, mailinglist, ... and import the old data from github, of course.

But as said, the tooling is not available for doing bug tracking in plain text and distributed, so I won't move.

To end this wall of text...

I didn't expect this article to get this long. But now it is. I hope I could explain what the purpose of imag is.

Feel free to ask me questions (via mail, you will find it) or suggest improvements to this article.

tags: #linux #open source #programming #rust #software #tools #imag

This is the 9th iteration on what's coming up in imag, the personal information management suite for the commandline.

First of all, I want to thank and welcome kbknapp who contributed a Makefile which enables to build the imag tools with just one command: make. Thanks a lot for that!

What happened

I wasn't able to do that much in the last 14 days, as my bachelors thesis wants some attention as well. But I was able to merge some nice PRs anyways, so here we go.

New issues!

In the last 14 days I tried to open as many issues as possible, tagging them and therefor making them accessible by possible new contributors (as some of them are tagged with the complexity/easy tag).

Diary!

Finally, the initial implementation of the diary module was merged! Yay! It already has some features I really like, but isn't feature-complete yet and there might be bugs. But the initial codebase is merged and I'm really happy about that.

One can use imag now as a personal diary. I did not integrate some diary tool into imag by now, because I thought implementing it myself would be kind of fun. Of course, I would like to add other tools as backend but I don't know of any (and I do not use any) I would find sufficient for integrating into imag.

More libraries and library-functionality!

Two awesome PRs were merged which added new functionality.

The first one is the “move” functionality for the store. It is not yet used by any other crate, but gives the possibility to save a copy of a store entry or move a store entry to a new location within the store. This functionality is not yet tested, but as we are pre-release anyways, this is not an issue in my opinion.

The other functionality is the libimagentrymarkdown which adds markdown-parsing functionality. There is just the initial codebase for this crate added by now, but the library will make it possible to parse the contents of a store entry to a web page and view it in the browser, for example.

The other merges

I merged some rather critical bug fixes, for example one were an entry wasn't edited correctly when calling edit_content() (from libimagrt) on it. A code-cleanup PR for libimagentrylink and some other fixes were merged as well.

The future

So lets see what's in the pipeline at the moment.

I'm working on a new crate called libimagref which gives access to primitives and functionality to reference files which do not live inside the store. For example to reference some music files, the libimagref can be used to generate a reference in the store which tracks the file(path), optionally having the possibility to check the metadata of a file by generating a hashsum over its contents and/or permission settings.

Some tests for the tagging binary (imag-tag) are work-in-progress.

For the next 14 days I hope to finish the libimagref implementation as well as getting some progress in the libimagentryfilter/lang PR, which aims for a DSL implementation in the entry filter crate, giving the user access to a filter language to filter out entries on the commandline like so (diary example here):

imag diary list --filter 'e.size() > 5.kilobytes() && e.lines().first().empty()'

Of course, the upper example is not how it will look like, as I'm still not sure what DSL implementation I will use and how I will model the access to the entry data, but the idea will be the same.

Another thing I really want to test out is quickcheck. I guess this could be really useful to test imag library crates properly and I hope to get something up in the next 14 days in this regard.

tags: #linux #open source #programming #rust #software #tools #imag

Another 14 days are over. Here's my update what's coming up in imag, the personal information management suite for the commandline.

The past

We had 40 closed issues and pull-requests in the last 14 days, which is really awesome.

I was able to merge 29 pull requests in the last 14 days, which is quite a lot, so I won't elaborate on all of them (some of them are just small fixes), but let me do a short iteration on the highlights:

• We implemented Store::get() which returns a None if the entry does not exist (Store::retrieve() implictely creates the entry).
• The Store::get() functionality was also implemented in the imag store, imag notes and imag tag commands, causing them to fail if you try to list or view non-existent store content.
• We have a new Hook which gets executed by the store after the store is loaded (so it is called only once, but hooks might do some fancy stuff before the user commands are executed).
• We have a new crate for writing Date and Time interfaces, which helps parsing user-specified date and time. This crate is not tested yet, though some tests exists within the crate and they succeed.
• Another crate was introduced for querying the user to provide a store id if he hasn't already, providing a fuzzy-search interface based on the interactor crate. This crate is not tested yet.
• The error crate libimagerror provides a way to define custom members and custom code for the error types now, beeing more flexible now.
• libimagerror also implements Into for the error kinds now, making the code using it even less complex.
• All crates were rewritten to use the infrastructure provided by libimagerror
• The binary at bin/imag was rewritten from Bash to Rust.
• Updates for some dependencies were done
• Bug fixes, including but not limited to
  • A bug in the config-finding algorithm in libimagrt
  • Bugs in the Hook execution in libimagstore
  • Duplication of data in libimagstore (FileLockEntry and Entry had the StoreId stored, despite the one dereferences to the other)

---

I also had (finally) some progress with my diary implementation. I guess I may finish it within the next 14 days. I'm serious this time, as the current code works and almost all the things are implemented at the time of writing.

I also have some (little) progress with the bookmark implementation, though I'm not sure whether I will implement this (rather) simple tool by myself or re-use existing solutions. On the one hand the problems are rather simple to solve with the existing imag infradstructure, on the other hand, I don't want to double work. I guess I will end up implementing it myself anyways, as I consider the (external-)link code part of the infrastructure and I don't want to depend on an external tool which might be or get unmaintained at some point in future.

I also have a new crate in my pipeline, which helps with Markdown parsing. So for the first step, we will provide Markdown only. But I will wrap this crate libimagentrymarkdown into a more generic one (libimagentrymarkup) to be able to write more backends for it (thinking of textile, asciidoc and so on, maybe by using a pandoc interface) and support other markup languages later on without hassle.

All in all, I really made some progress with the project in the last 14 days. I wrote more than 80 commits in the four days of the GPN last weekend, which was a huge amount of the work in the last two weeks, of course. I hope I can continue my streak.

The future

Well, the future. I don't really think I can keep up with my latest streak, though I really hope getting the diary implementation done. Bookmarks would be nice as well.

One thing I'm thinking about almost constantly in the back of my head is the DSL implementation for libimagentryfilter. I really want to dig in a little deeper into this. Maybe I can get something working within the next 14 days and if it proves useful I might extend the DSL so I can script the Store with it. Of course, this won't be a replacement for Rust as implementation language, but might prove useful for smaller scripts and the like? I'm not sure.

I also hope to find some contributors for the project. If you are looking for rust projects to contribute to, feel free to ask me questions or simply try to implement one of the complexity/easy tagged issues on github. Of course you can also ask questions there, I'm happy to answer them!

tags: #linux #open source #programming #rust #software #tools #imag

Is it more important to learn things or to have progress? And especially, is it more important to have progress in an open source project or is it more important that people learn new things?

I got a pull request for my open source project imag which wasn't well-formatted. As we are all nice people in the rust community (not only because, but also because we have this code of conduct which basically says you should be nice to all people) I asked the contributor to clean up the history by rewriting it.

Basically, I had two issues with the PR:

1. The blank lines at the end of the files were removed. Some editors do this by default, but I don't like it as it makes diffs which add content at the end of a file rather complex (visually). If you have the blank at the end of the file, the diff is just one block which adds content, normally.
2. I asked to rewrite the commit messages but after this was done, only one big commit was left. This is not preferred style in my project, I'd like to have more small commits than big monolithic ones.

So I asked kindly whether the contributor cared to fix up the issues. He did! Awesome! But he also told me that he was new to git and didn't knew how to do it. So I told him the steps which needed to be done, of course. He got stuck at some point (yes, git can be challanging, I know that) and asked for more help while apologizing for the slow progress.

And that made me start thinking about whether it is more important to have progress or that people learn new things. The issue which was adressed by the pull request described above is a rather simple one. It is literally a no-brainer. For me. But maybe not for others. Maybe also for others, but the review process isn't and the “Please rewrite the history like this” request was the challange. Either way, there was a point where the contributor had to learn something new which caused the pull request to slow down.

Is it worth it?

I honestly think yes. Learning new things is way more important than getting progress in this kind of project. I myself had to learn a whole bunch of things before this project was on the way it is right now. I had to learn that in Rust, you have to add more types to cover complexity and the more types you have, the better is your setup and the more extensible it gets. Of course this is not always the case, but I would say mostely.

I had to learn a lot of things. I had to learn to fail and to retry, which are two really important things. I failed so hard, I had to restart my whole project (which had no contributors by then, so no problem).

At the time of writing, the pull request is open for 8 days. It touches about 100 lines of code by adding missing calls. It could be done by myself within 3 minutes. But in my opinion, it is way more important that we all learn something and have fun. I offered to do it myself to the contributor, but clearly stated that if he wants to learn new things, he is free of taking his time and trying out how to fix the pull request. I really hope he learns something and starts contributing more often because of this.

tags: #software #tools #programming #open source

Welcome to the 7th post about the last 14 days in imag, the personal information management suite for the commandline written in rust.

We had 22 merges in the last 14 days, 24 in the last 15 days, since the last article of the series.

Lets see what happened.

Merged pull requests

Next goes a short explanation on each of the merged pull requests from the last 14 days.

#339

The StoreId Type was rewritten and isn't a pub type StoreId = PathBuf anymore. This change should be fully backwards compatible, so nothing will change, though we have the possibility to implement custom traits on the type from now on, which can make things easy in some situations.

#385

This merge fixed a typo in the Readme file. Thank you, Ron, for contributing to imag!

#349

The libimagstore got a Walk Type which can be used to walk whole store directories. It is based on the walkdir crate and also iterates over directories while recursing into them as well. So one can use this functionality to explore the collections within a module storage.

#387

This merge removed some old code which did some configuration file checking but was wrong.

#389

This merge added an example configuration file to the repository.

#388

This was also a bugfix which fixed some missing error handling in the configuration file parsing code.

#390

This was a mass-update of the log crate dependency for all crates in the repository.

#393

With this merge we updated the url dependency of libimagentrylink

#394

We made a helper function which could be used to filter links to external link files public with this merge.

#395

This was one of the greater PRs.

With this merge, we made the StoreIdIterator generic. Before this merge, the StoreIdIterator type was a wrapper around the Paths Iterator type from the glob crate. This wasn't really good, as we couldn't use the StoreIdIterator type to iterate over a Vec<StoreId> for example. This merge makes this now possible.

#384

llogiq made some improvements to the imag code base by running clippy on it and removing the warnings clippy yielded. Thank you very much for contributing to the imag codebase!

#401

General code cleanup in libimagutil.

#404

With these patches merged we now have the possibility to specify multiple tags when using imag-tag to add or remove tags.

#402

General code cleanup in libimagrt.

#403

General code cleanup in libimagstore.

#412

We have now colored log output with libimagrt! Awesome!

#411

Some improvements to the CONTRIBUTING.md file.

#413

Error tracing prints the errors now in red (blinking). I really like that!

#410

A new library got into the imag codebase: libimagerror contains code to handle imag error types.

#415

This is a follow-up patch for #410 to move the trace_error() functions from libimagutil to libimagerror.

#417

This merge added a function in libimagrt to generate the Runtime setup or exiting on failure. Just convenience and less boilerplate code in the binaries. Yay!

#418

Here we added some helper functions in libimagentrytag for parsing the commandline options.

#422

matthiasbeyer/libimagerror/generate-module

Another follow-up PR for #410. With this merged, we generate a full error module when using the macros from libimagerror. This way, we have more flexibility in generating our error code.

#416

Aaaand another follow-up PR for #410. With this merged, we can call into_error() on all error kinds to generate the appropriate error object out of them and into_error_with_cause(...) to pass a cause. This will lead to more readable code everywhere.

Maybe another follow-up will follow which then removes the noisy FooError::new(FooErrorKind::SomeErrorKind, Some(Box::new(other))) and simply put FooErrorKind::SomeErrorKind.into_error_with_cause(Box::new(other)), which is way more readable IMHO.

What's coming up

I hope we can continue these libimagerror patches and add the ability to add custom code in the error types. We need this for #370 as we add some custom members in the HookError Type in this PR.

I'm not really sure what else will be done in the next 14 days. I am still working on the diary implementation (actually with little to no progress in the last 14 days). Because of this I'm not sure what I can get done in the next 14 days.

I really hope to get some more tests merged. One PR (tests for imag-tag) is already pending. I also want to get some markup-parsing library working (I have a pending PR for this which is already three months old).

If you think you could contribute some code to imag, feel free to do so, there are a lot of open issues which are tagged easy!

tags: #linux #open source #programming #rust #software #tools #imag

Two new contributors! Yay!

Another 14 days in imag, lets start iterating over the latest changes (spoiler: there are not that much).

The past

As said, there are not that much changes in imag in the last 14 days. Why is that? Because I have a whole lot to do for my university and some events I'm organizing. The event takes place this week, so after that I hope I can focus more on imag (and my bachelors thesis, hopefully).

But hey, anyways, things happened. Lets iterate.

Fixes!

We had some fixes in the last 14 days and some more are in the pipeline. We removed some result.err().unwrap() calls in the error handling part of functions and replaced it with result.unwrap_err(). Kind of cosmetic, but anyways an improvement in my opinion.

Testing

I merged a PR for imag-link testing. Still, we are testing by using bash scripts, which is what I really want to improve. I want to rewrite the tests in Rust to be more safe.

Hooks

I merged a new hook: A hook to verify links in the entries which are about to be altered or touched by a store call. You can use it as pre-update hook or as pre-retrieve hook.

The hook verifies that the entry which is about to be touched links to entries which actually exist on the filesystem.

Warnings

I removed almost all warnings which occured when compiling the imag codebase. This is awesome, isn't it? I enabled some lints and compilation will now fail if someone screws up. I love it!

The future

I got some PRs. Awesome!

PR 383 is a PR for implementing libimagtask and imag-task, using task-hookrs as backend to talk to taskwarrior. I will, of course, support this as much as possible with both patches and suggestions.

PR 384 is a PR from llogiq, one of the authors of clippy. It fixes some issues with the codebase overall. So awesome!

These PRs will be merged as soon as they are ready.

I'm not sure what else I can do in the next 14 days, but I want to go on implementing the diary and maybe get the filter language working, which is a rather complex task. Maybe I will implement the bookmarking module by adapting another tool or maybe by implementing it myself.

tags: #linux #open source #programming #rust #software #tools #imag

Wow, another 14 days! When I started writing this article, I thought I didn't manage to do a lot of changes in the last 14 days. git proved me wrong.

What happened

When I started writing this article, I thought “Oh man, another 14 days with little progress in imag”. I was proven wrong:

$ git log --oneline --merges --since=2016-04-07
c3618ec Merge pull request #343 from matthiasbeyer/libimagstorestdhook/flock
45f93e5 Merge pull request #366 from matthiasbeyer/update-dependency/regex
25a71ce Merge pull request #365 from matthiasbeyer/libimagstorestdhook/zero-warnings
a2288ba Merge pull request #364 from matthiasbeyer/libimagnotes/zero-warnings
791b3ad Merge pull request #363 from matthiasbeyer/libimagentrylist/zero-warnings
f0d29da Merge pull request #362 from matthiasbeyer/libimagentryfilter/zero-warnings
e040d0d Merge pull request #361 from matthiasbeyer/libimagrt/zero-warnings
3e29ec0 Merge pull request #359 from matthiasbeyer/remove-init-logging
19808cf Merge pull request #360 from matthiasbeyer/update-dep/iterools
99e1e41 Merge pull request #353 from matthiasbeyer/rename/libimaglink
a5e9bf7 Merge pull request #358 from matthiasbeyer/imag-view/zero-warnings
d3b94df Merge pull request #354 from matthiasbeyer/libimaglink/zero-warnings
3a5c965 Merge pull request #355 from matthiasbeyer/libimagutil/zero-warnings
86ae438 Merge pull request #356 from matthiasbeyer/imag-counter/zero-warnings
f64ed8e Merge pull request #357 from matthiasbeyer/imag-link/zero-warnings
ed6d912 Merge pull request #352 from matthiasbeyer/rename/libimagtag
179ae2b Merge pull request #350 from matthiasbeyer/libimagtag/zero-warnings
f003625 Merge pull request #351 from matthiasbeyer/libimagcounter/zero-warnings
ff20e79 Merge pull request #338 from matthiasbeyer/imag-link/external-linking
e4e3c05 Merge pull request #326 from matthiasbeyer/libimaglink/external-linking-rewrite
3d124df Merge pull request #347 from matthiasbeyer/libimaglink/unique-internal-links
17610ef Merge pull request #337 from matthiasbeyer/imag-link/list-implementation
20a76f8 Merge pull request #330 from matthiasbeyer/update-contributing
535483d Merge pull request #335 from matthiasbeyer/libimaglink/bugfixes
18627f1 Merge pull request #336 from matthiasbeyer/imag-link/cli-unpacking-bugfix
8d7e2d9 Merge pull request #331 from mario-kr/add-description_to_build/run_imag
9866c63 Merge pull request #311 from matthiasbeyer/libimaginteraction/init
92b471b Merge pull request #328 from matthiasbeyer/imag-link/fix-ui
cd2bc0b Merge pull request #325 from matthiasbeyer/libimaglink/internal-linking-use-storeid

Let me do a short iteration.

Zero Warnings

I opened an issue about warnings and that we should enable compiler lints to make them hard errors. Then I started to write pull requests for each crate in the repository and also got them merged. We have almost no warnings left when compiling imag, which is what I consider nice progress!

Init-logging

Some time ago we rewrote the Runtime setup code for logging, so we had the logger enabled at the earliest possible point in program execution. Now I removed some debug!() calls which were done in the main.rs of almost all crates right after the Runtime object was built which stated something like "logging enabled...". These debug!() calls were outdated, so I removed them.

Libraries renamed

Some of the libraries were renamed. For example "libimagtag" got renamed to "libimagentrytag".

This is none of a big deal and as we are pre-1.0 I can do this, I guess.

External link rewrite

This is one of the gems here, really. I finally got my feet up to rewrite the external linking part of the linking library and got imag-link adapted so it works appropriately now. How awesome is that?

More Hooks!

I implemented a flock() hook in libimagstorestdhook and I have another open PR for a verify-link hook which uses libimagentrylink to verify that all linked entries exist.

Dependencies updated!

I updated a number of dependencies (namely itertools and regex) and I also rewrote the Cargo.toml files to depend not on the master-minor-patchlevel version of the crates but on master-minor versions, so cargo can figure out what patchlevel to use.

What's coming up

I'm about to re-write the libimagdiary because what I did in my initial PR for the library was a mess. I tried to implement the diary functionality by re-using the note functionality, which resulted in ugly spagetti code and the like, so I decided it would be better to just re-write the whole thing.

I'm about to merge a PR which introduces the concept of non-failing hook errors. So a hook can return an error which does not abort the store operation which invoked the hook itself.

I also started to implement libimagentryview on my own, as the friend of mine who said he wants to do it hasn't made anything yet, so I'm a bit pissed of on him. But Hey, it is open source, nobody can force anybody! So I will just stick with implementing it on my own (and it is also almost finished).

I also started to refactor the implementation of the StoreId type, as we might need to extend this type in external crates, so doing a pub type StoreId = PathBuf was not the best idea and I started to rewrite it as pub struct StoreId(PathBuf) – I hope I can manage to do this in a backwards compatible way, but I'm not sure.

I also started implementing a Walk type for the Store which does a walk through all the objects in the store. This is not an iterator but something that also notes folders within the store and gives the possibility to iterate through a tree rather than through a flat list of entries. Could be helpful sometimes.

We also have an open PR for DSL support with libimagentryfilter, so the user is able to filter entry iterators with a DSL. This is early-state but looks rather promising IMHO.

tags: #linux #open source #programming #rust #software #tools #imag