Proposals (Political/Workflow) #4460

Minion Design

Added by Steven G. almost 3 years ago. Updated almost 3 years ago.

Status:NewStart date:02/25/2012
Priority:NormalDue date:
Assignee:-% Done:

0%

Category:-
Target version:-

Description

I'm just gonna place this here as a proposal for discussion, so there might be several issues in one but I feel it's worth having it centralized and discussed first (agreed or disagreed on) before going treating them individually.

Okey so the way I understand it and from what I've seen of it Minion is basically this hybrid console/web based thing that auto detects Task_* classes and is able to run them.

Personally I disagree with having a console module double as a web interface. If there's some web or whatever interface to run Tasks then it should be a separate module. There are a few reasons for this but mainly it's due to the requirements for console and browsers and whatever else are different and the behavior is different as well. So...

View files Vs Functions

From what I've seen the console uses View files as output. Here's the problem with that.

First of all consoles unlike browsers don't handle newlines, empty spaces, etc for you. You might appear to be able to get away with it because git has a feature that translates the newline characters to your systems default but if someone were to physically download a copy of it then someone on one of the 3 standards for new line is likely going to see it wrong. Because the new lines in your file are not necessarily the system default new line, ie. PHP_EOL. Similarly you might appear to get away with wordwrapping when you only print a 3 word sentence but once you got something that's got a few words in it you're suddenly in trouble.

I'm not saying you shouldn't use view files, it's just that whenever you want to produce pretty output on a console you have to go though the process of formatting it yourself, so it's usually something like 70% logic, 29.9% stuff you pulled from a configuration file, 0.1% stuff you actually printed out (from personal experience) unlike a typical view where you have markup to worry about. It's basically like with a calculator, you wouldn't create a view file to output the formatted number you would just send that as the response.

The thing to note here is that a lot of the processing you do on your output you would obviously do though functions. Because lets face it you're going to do the same formatting operation 2-3 times why wouldn't you turn it into a function. So let's take the calculator example, the first call is equivalent obviously. If you call a function to output it, or call a view file to output it's pretty much equivalent. The problem comes when several other functions are required after the call (like say some word wrapping procedure). If you have it in a function it's pretty natural, if you have it in a view file however, it's really wierd: you have to call other view files to get their rendering function or duplicate the code, and so on and so forth. And treating view files as "rendering functions" produces really awkward code. And no it doesn't make sense to have a hybrid view / function, since if you're going to send the user to functions then make the initial call a function too so the user doesn't have to navigate all over the place to see the logic.

There's also the problem of what tasks are and what console output is. Console output is more or less log output, and task's are this kind of controller/model hybrid in a way as far as their relationship with the console works. I say that because you expect direct feedback on execution. Executing something and only displaying the full log afterwards is not acceptable, it needs to be when it happens. So hence another problem with using views is you end up calling a ton of very small and insignificant ones constantly.

And then there's the issue of how console output just like on a console should have a silent switch or simply display only to the console.

Just my 2cents if you want to stick to views by all means. Just highlighting the cons.

Console format

For minion looks something like this:

minion --task={some:task} --something={blah-blah}

First the task is always going to be first parameter and it's always going to be there, so the whole --task part is redundant. You should just follow the "command" style where each task is known as a command and flags follow after it.

ie.

minion some:task --something={blah-blah}

Secondly, the whole --something={blah-blah} style is just working against the way consoles are meant to be used. Imagine for a second I had a string with spaces in it. Oh look time for you to add support for --something="some string" and hope nobody uses --something='some string' or some other style. What about if I have a = sing, --something==something? maybe I actually meant --something="==something"? Confusing isn't it. Oh, what if I want a list? what is that in stupid quotes too? ie. --something="joe fred earl"? Err. Oh and better be careful with those quotes never know what might happen on some consoles if you write --something="my text " --something-else="my "

The simple standard way of,

--something Text --something-else " Test Test " --some-list 1 2 3 4 5 6

Just works and it's simpler code too.

Console structure

The current format looks something like this:

 * some:task
 * some:other:task

And the help pages for each command are optional.

Okey let's see...

The COMMANDS page...

Needs to contain syntax help, usage example, and how to get help information (by default) and a list of commands with clear and crisp descriptions for each.

Example: http://pastie.org/private/ybbpi3uzrjf0iwzosqrqda

The COMMAND pages...

Need to be MANDATORY! You need to understand you can easily get 20-30 commands and there's no way in hell you'll know the exact flags for all of them. If a command does not provide the minimal required information it should not EXIST. And all commands should be displayed in the same exact format, it makes it easy to navigate though the pages and easy to understand, saving time and effort. It also makes it so you know what to expect of the help page of each command.

Example 1: http://pastie.org/private/7ha5b71v70eztu0pn1j2hw
Example 2: http://pastie.org/private/zscowvjjaweuggtemffxa

Commands having dynamic flags...

Horrible idea. Basically I have to look into the code of the task to know it's flags. And don't even think of comparing it to calling a method/function, a console is not a IDE! you don't get no help on parameters in the console; there's no magical navigation aids either.

Each command needs to be defined, have a description, if it has a default value that needs to be displayable in the console. It also needs to allow for shorthand, since really typing --really-long-and-obnoxious flags can be a real bother. Requiring to know what the flags for the command are, or what that flag you never used before was, is pretty common.

Help directives

The "help" command should be reserved and just work by default; since it should ALWAYS exist. For command help I recommend the "help [command]" syntax over the "[command] --help" syntax since it follows easier from "help", ie. you just type up then type your command, whereas for the --help variant you have to go into the middle of the line and type the command between. Same problem applies to editing as well, ie. going from "help command1" to "help command2" is probably easier and more intuitive (no using the home key doesn't make the other one just as easy since there's the "minion" part and probably other things).

Support for Color, Password, etc

They are (a) not cross platform (b) make it so tasks can't be called outside a console environment (c) can potentially create ambiguity, like for example the excessive use of color. The console applications that need these things are very specialized and IMO outside the scope of what the tool should do. Having things as text and flags is also a lot more useful since they can be used with and by other commands.

There's also the issue with the code you're currently using of potentially just breaking the console I think.

Migration

Okey so I'm looking at the interface for migrations and I'm not quite getting how it actually handles CONSTRAINTS on the database. So obviously while testing the concept I went and created two Migration's, one for User and one for some general structure. To avoid overly complicated explanation let's pretend the structure was that of ye' ol' stupid blog so posts with comments. So I have 2 along with 2 "migration" configuration files in each module (the structure one and the user one). The configuration just sets various customization options like how to handle indexes and so forth and registers the migration class.

So now I create a db:install task obviously and run it. The question is, how exactly does one set constaints on the model's you know when the order is not defined? Traditionally if I just have a install script the constraint just go at the bottom and that's that but with the current up / down structure you suggest the constraints would have to go in up. However while the structure part might know of the existence of Model_User and hence ask for it's table it can't set a constraint on it before the migration for user went up as well.

Obviously something like what Vendo does would work where you just have this monolith single migration with everything. But if you do it like that, what's the point really? Might as well just stick it in install.php and be done with it. As I see it if I were to include a module into the system and it had a migration it should work just like that.

One way, while testing, I found to go around the problem was to have a finalization method in the interface and have the up method return a callback. The install task would then call the callbacks after all the up's and the callback functions would do the constraints on the database.

So... any way to do this with just up / down?

Default commands and structure

I've went into detail over this in another issue, but basically the way kohana structures it's initialization needs to be revised. Specifically related to CLI tools such as minion, the framework's initialization needs to be split from index.php into a seperate file (probably init.php for naming consistency) and index.php just require it and execute the request. The CLI tool would then call it as well and execute the request; that way code isn't duplicated and there's no dirty code such as now where minion does some wierd ../.. and assumes I actually use the default structure, which I don't... This seperation would probably be useful for other tools too.

As for default commands, currently there are none. But I can easily think of a few such as a command for generating configuration overwrites (ie. get all configs, merge, dump in new file) or a command for generating cookie salt and such, a command for installing so that any module can just have a migration and config and it's just pluggable into the app, etc.

History

#1 Updated by Isaiah DeRose-Wilson almost 3 years ago

What version of Minion are you talking about? I recommend looking at the official minion module that will be included with Kohana 3.3 (see https://github.com/kohana/minion/).

It sounds like there is some confusion (tasks are separate from controllers now, there is no web interface, etc) on your part about how minion works. Some of your suggestions are valid, but I think it would be helpful if you reviewed the minion module above and updated your feedback based on how the latest version works.

#2 Updated by Steven G. almost 3 years ago

Isaiah DeRose-Wilson wrote:

What version of Minion are you talking about? I recommend looking at the official minion module that will be included with Kohana 3.3 (see https://github.com/kohana/minion/).

It sounds like there is some confusion (tasks are separate from controllers now, there is no web interface, etc) on your part about how minion works. Some of your suggestions are valid, but I think it would be helpful if you reviewed the minion module above and updated your feedback based on how the latest version works.

The way I look at this...
https://github.com/kohana/minion/blob/3.3/develop/classes/Controller/Minion.php
https://github.com/kohana/minion/blob/3.3/develop/init.php

You have another way of calling the CLI inteface though the routing / controller. So supposedly it's only for using with the CLI. But why isn't the Route conditioned to Kohana::$is_cli? And why even have this feature to begin with? If you're only running from the CLI you already have access to the normal way of doing it, and it's not like you'll have access to php index.php but don't have access to php minion.

Anyway, my point was this is more of a double standard more so then the specifics.

I think it would be helpful if you reviewed the minion module above and updated your feedback based on how the latest version works.

The issue text above? Let's say it's uneditable.

#3 Updated by Jeremy Bush almost 3 years ago

Those two files aren't supposed to be there. They need to be removed before release, they were left over from the port. Minion has nothing to do with controllers or routes anymore. I've removed them.

Needs to contain syntax help, usage example, and how to get help information (by default) and a list of commands with clear and crisp descriptions for each.

This is what the --help flag is for. It already does all this for you. If you think it can be improved, patches welcome.

They are (a) not cross platform

If they don't work, they don't work. No big deal. In *nix, they'll work pretty much all the time, which is all that really matters.

make it so tasks can't be called outside a console environment

Tasks shouldn't be called outside a console environment anyway.

Okey so I'm looking at the interface for migrations

Migrations are a separate module that isn't supported by Kohana. Minion is simply a console task runner.

the framework's initialization needs to be split from index.php into a separate file

You mean bootstrap.php? :)

As for default commands, currently there are none. But I can easily think of a few such as a command for generating configuration overwrites (ie. get all configs, merge, dump in new file) or a command for generating cookie salt and such, a command for installing so that any module can just have a migration and config and it's just pluggable into the app, etc.

Yeah, we have no plans for any of these in the initial release, but they might be useful as minion matures.

First the task is always going to be first parameter and it's always going to be there, so the whole --task part is redundant.

--task isn't required, you can pass it like `./minion task:subtask`, which is the convention.

Secondly, the whole --something={blah-blah} style is just working against the way consoles are meant to be used.

It's actually standard unix syntax. Your point about whitespace is taken, and I'd consider that a bug (you currently can't pass a parameter with a space in it, even in quotes). Minion should follow standard unix behavior if at all possible.

#4 Updated by Steven G. almost 3 years ago

Jeremy Bush wrote:

Needs to contain syntax help, usage example, and how to get help information (by default) and a list of commands with clear and crisp descriptions for each.

This is what the --help flag is for. It already does all this for you. If you think it can be improved, patches welcome.

The emphasis was on "description" not "list". :)

As for patch. I actually made an alternative version; well "experimented" with an alternative version at this point. (The previous examples were simply output from it.) Unfortunately I don't know about patches... sadly after forking and messing around with for 30min (ie. customizing it, ie. removing stuff I didn't want) I ended up with something like an empty CLI class and 2 empty methods. But I still liked the idea of it so just remade it from scratch.

I'll post it in an example repository for reference....... as soon as I rip namespace out of it, change configuration calls to what kohana uses, etc so it will work with a default kohana. Shouldn't take too much time.

They are (a) not cross platform

If they don't work, they don't work. No big deal. In *nix, they'll work pretty much all the time, which is all that really matters.

There are additional benefits to NOT using view files and using configuration files instead.

Say you make a Task. But it's not good enough for my needs. So I extend your task and add a extra flag. Now the next logical step is to add documention for that flag. With views I have to REPLACE your view, OR not document it. A lose-lose situation either way, since I either have to maintain and re-check for changes you make to it OR hide my own changes.

With configuration that's not an issue. Since assuming the documention paragraphs (ie. initial short description + explanation, examples etc) are represented as an array, which they should, all I have to do is create a configuration entry in the module where I "improve it" and add the new documention along with the flag information, and it will magically merge into your documentation and give me the complete documention pages. :)

make it so tasks can't be called outside a console environment

Tasks shouldn't be called outside a console environment anyway.

This is kind of iffy. On one hand I agree with you, if you want to call a task you should (in an ideal world) just create a function somewhere then wrap a task around that function and when you need the functionality you just call the function. HOWEVER, there are these pesky commands like database commands, such as say a db:migrate command, or a db:install command. Those things could take a while to execute at the same time getting by-the-second feedback is pretty relevant (so they have to be Task's), especially if they happen to fail. You wouldn't be running them every day but it would be nice for the cases when you do need to run them for you to just be able to execute the task via a web request or something so you don't have to duplicate code. Things like a fancy install interface, or fancy update interface come to mind.

Is a little code travesty worth it? Alternative would be duplicating the install script code.... which do you think is the lesser evil here?

Okey so I'm looking at the interface for migrations

Migrations are a separate module that isn't supported by Kohana. Minion is simply a console task runner.

I see. (for reference) Is kohana against supporting some (form) of them? Or is it just nobody has bothered with them.

the framework's initialization needs to be split from index.php into a separate file

You mean bootstrap.php? :)

I don't know if sticking it in there is a good idea. It creates all these confusing ../.. paths and weird referencing. I don't think a file in the route of next to minion and index.php is that much of a problem.

Secondly, the whole --something={blah-blah} style is just working against the way consoles are meant to be used.

It's actually standard unix syntax. Your point about whitespace is taken, and I'd consider that a bug (you currently can't pass a parameter with a space in it, even in quotes). Minion should follow standard unix behavior if at all possible.

I don't remember there being such a "standard unix behaviour" =P the standard unix commands are pretty loose in how the handle flags as far as I'm aware of--they don't all have usage documentation for nothing. What are these commands that make this common? I can see it as potentially being an issue of the commands in question not actually requiring support for the extra cases or dealing with is as with most commands in the command [all the flags in some form or another] "your input" pattern which works for most....... but only when you have 1 input. But you can easily have multiple input. A new:user command for example: name, password, already double input.

Also available in: Atom PDF