Differences

This shows you the differences between two versions of the page.

Link to this comparison view

irc:1456873200 [2017/05/27 13:44] (current)
Line 1: Line 1:
 +[05:53:54] *** ChanServ sets mode: +o temporal_
 +
 +[14:37:55] <​Narigo>​ temporal_, I'd like to understand how releases are done in the official modules/​clients/​services... and maybe how we can make the release process faster / how versioning is done, etc.
 +
 +[14:38:20] <​temporal_>​ Narigo what do you want to do exactly ?
 +
 +[14:38:37] <​Narigo>​ and then release another version of the async mysql-postgresql to have the latest fixes available for others ;)
 +
 +[14:38:44] <​temporal_>​ I think we can definitely do that
 +
 +[14:39:02] <​temporal_>​ basically I think that the main version should align with vertx stack
 +
 +[14:39:04] <​Narigo>​ currently the version matches Vert.x itself
 +
 +[14:39:07] <​temporal_>​ i.e 3.2 -> 3.2 etc...
 +
 +[14:39:14] <​temporal_>​ but we could have a 3.2.2 of mysql
 +
 +[14:39:32] <​temporal_>​ like I'm planning at 3.2.2 of vertx-lang-ceylon based on Ceylon 1.2.2
 +
 +[14:39:45] <​temporal_>​ specially for projects like mysql or ceylon
 +
 +[14:39:52] <​temporal_>​ no other modules depends on this one
 +
 +[14:40:00] <​Narigo>​ what if i introduce a breaking change to the mysql project, but it would still work on vert.x 3.2?
 +
 +[14:40:03] <​temporal_>​ and if someone wants a newer version in its stack
 +
 +[14:40:20] <​temporal_>​ no, in this case I think it should wait 3.3
 +
 +[14:40:28] <​temporal_>​ or find a different naming
 +
 +[14:40:32] <​temporal_>​ versionning I think
 +
 +[14:40:34] <​temporal_>​ I mean
 +
 +[14:40:41] <​temporal_>​ that being said
 +
 +[14:40:55] <​temporal_>​ as mysql connector is "tech preview",​ it could be ok if thats not too disruptive
 +
 +[14:42:15] <​Narigo>​ Then I think we need a general discussion about versioning that works for all projects and lets users understand what it means...
 +
 +[14:43:21] <​temporal_>​ I think there is an "​implicit"​ versionning scheme so far
 +
 +[14:43:22] <​Narigo>​ maybe a 3.2_1.0.0 with 1.0.0 is (for example) the mysql modules version and 3.2 being the vert.x version needed...
 +
 +[14:43:32] <​temporal_>​ meaning what we have followed so far
 +
 +[14:43:41] <​temporal_>​ (which is basically what lot of OSS project do)
 +
 +[14:43:52] <​temporal_>​ ok I see what you mean
 +
 +[14:43:58] <​temporal_>​ it's going to be funny discussion :-)
 +
 +[14:44:14] <​temporal_>​ but yes I agree with you we should have a versionning scheme that
 +
 +[14:44:23] <​temporal_>​ make the whole stack concept coherent
 +
 +[14:44:38] <​temporal_>​ and yet have some flexibility for doing such thing like you describe
 +
 +[14:44:43] <​Narigo>​ I'd like to have semantic versioning actually - at least for my project. But if it's just that, we'd loose information about the Vert.x version
 +
 +[14:45:09] <​temporal_>​ well, 2 hours ago I've seen Gavin King ranting about semantic versionning :-)
 +
 +[14:45:12] <​temporal_>​ on twitter
 +
 +[14:45:25] <​Narigo>​ what did he say? :)
 +
 +[14:45:51] <​temporal_>​ "I know this is going to hurt your feelings but: sorry, Semantic Versioning is a cargo cult. It[unknown:​rsquo]s about emotions, not engineering."​
 +
 +[14:48:22] <​Narigo>​ ...and I can't follow/see the logic behind that statement ;)
 +
 +[14:49:34] <​Narigo>​ I know there are problems with it, when someone doesn'​t correctly use it (we're all humans and may introduce breaking changes in a patch) - but what's the alternative?​ :/
 +
 +[14:50:32] <​temporal_>​ actually I don't know much about semantic versionning
 +
 +[14:50:55] <​temporal_>​ so if that's something we are going to discuss, I would need to spend time
 +
 +[14:51:11] <​Narigo>​ Well, it's pretty simple, actually... <​breaking>​.<​feature>​.<​patch>​
 +
 +[14:51:16] <​temporal_>​ yes the problem is that I think
 +
 +[14:51:31] <​Narigo>​ ...in theory ;)
 +
 +[14:51:38] <​temporal_>​ it prevents to have a concept of stack
 +
 +[14:51:49] <​temporal_>​ and a concept of new features independant of this stack
 +
 +[14:51:53] <​temporal_>​ at the same time
 +
 +[14:52:10] <​temporal_>​ and you want breaking changes right ?
 +
 +[14:52:14] <​Narigo>​ how about <​stack>​.<​breaking>​.<​feature>​.<​patch>​ ?
 +
 +[14:52:14] <​temporal_>​ in api
 +
 +[14:52:53] <​temporal_>​ it would mean that we could have breaking features in the stack :-)
 +
 +[14:53:01] <​Narigo>​ not right now - but maybe in the future... you never really know when you need to introduce a breaking change :D
 +
 +[14:53:07] <​temporal_>​ perhaps one idea would be to do like linux kernel
 +
 +[14:53:56] <​temporal_>​ but anyway you want breaking changes in mysql driver right ?
 +
 +[14:54:03] <​temporal_>​ as it is tech preview we can change the api
 +
 +[14:54:10] <​temporal_>​ that's what tech preview is made for
 +
 +[14:54:27] <​temporal_>​ for example in 3.1, Vert.x shell was in tech preview because we knew it would have things changing
 +
 +[14:54:40] <​temporal_>​ and right now mysql has been in tech preview, so it allows api changes
 +
 +[14:55:24] <​temporal_>​ so the only issue would be rather to find a version for this change if you want it based on Vert.x 3.2.1
 +
 +[14:55:40] <​temporal_>​ because it could not be released as async-mysql-postgres 3.2.2
 +
 +[14:55:47] <​Narigo>​ I'm not sure if I need breaking changes right now. I just want to say "this should be okay to use" and not "this is a preview and we will change it all the time and never ever give you a backport, even if necessary"​ ;)
 +
 +[14:57:13] <​temporal_>​ perhaps you coul try to do without breaking changes and release it as 3.2.2
 +
 +[14:57:21] <​Narigo>​ I mean, don't the other projects may have the same problem?
 +
 +[14:57:35] <​temporal_>​ other vertx proejcts ? or outside ?
 +
 +[14:58:00] <​Narigo>​ well, JDBC driver for example
 +
 +[14:58:13] <​temporal_>​ to be honnest
 +
 +[14:58:15] <​Narigo>​ other projects like async mysql-postgresql
 +
 +[14:58:28] <​temporal_>​ we always break a couple of minor things in releases
 +
 +[14:58:34] <​temporal_>​ this is *ok*
 +
 +[14:58:38] <​temporal_>​ as long as it is small changes
 +
 +[14:58:40] <​temporal_>​ and documented
 +
 +[14:58:59] <​temporal_>​ https://​github.com/​vert-x3/​wiki/​wiki/​3.2.0-Breaking-changes
 +
 +[14:59:12] <​temporal_>​ we recknognize that things are not perfect
 +
 +[14:59:17] <​temporal_>​ and we prefer to have a small change
 +
 +[14:59:32] <​temporal_>​ that is acceptable for use
 +
 +[14:59:36] <​temporal_>​ as long as we don't abuse it
 +
 +[14:59:45] <​Narigo>​ Semantic versioniong
 +
 +[15:00:03] <​Narigo>​ Semantic versioning should document breaking changes :)
 +
 +[15:00:41] <​temporal_>​ I'm going in a 1h training now
 +
 +[15:00:43] <​Narigo>​ (maybe "would document..."​ is more appropriate)
 +
 +[15:00:52] <​temporal_>​ (migrating from zimbra to gmail :) )
 +
 +[15:01:02] <​Narigo>​ Have fun :)
 +
 +[15:02:23] <​Narigo>​ It's just to start the discussion and maybe we can all think about it and come to a good solution that works for all the projects, stack, whatever we have... ;)
 +
 +[18:12:51] <​SmilSenil>​ Hi, which Github repo contains vertx3 core?
 +
 +[19:01:21] <​jackhill>​ SmilSenil: I think this is it: https://​github.com/​eclipse/​vert.x/​
 +
 +[21:38:14] <​jtruelove_>​ temporal_ when you deploy a verticle from a verticle does it share the same event loop aka context or does it get its own?
 +
 +[21:38:26] <​temporal_>​ it gets its own
 +
 +[21:39:10] <​jtruelove_>​ that's what it looked like from DeploymentManager just wanted to check
 +
 +[21:39:57] <​jtruelove_>​ and by default all deployed verticles are non-blocker unless you explicitly mark them as '​worker'​
 +
 +[21:41:23] <​temporal_>​ yes
 +
 +[21:41:46] <​jtruelove_>​ but the event bus is thread safe to use from the worker verticle?
 +
 +[21:41:52] <​jtruelove_>​ for message passing
 +
 +[21:43:50] <​aesteve>​ that's something I never fully understood : the threading model & the event bus
 +
 +[21:44:19] <​aesteve>​ for instance, if you're consuming messages within a worker vertical, what happens ?
 +
 +[21:44:31] <​temporal_>​ jtruelove_ yes it is safe
 +
 +[21:44:47] <​temporal_>​ jtruelove_ however you can have quirks
 +
 +[21:45:18] <​jtruelove_>​ lol my favorite, i've generally just written NIO '​worker'​ verticles
 +
 +[21:45:32] <​temporal_>​ jtruelove_ for instance if you send a message and wait with a reply handler , then block for some reason
 +
 +[21:45:54] <​temporal_>​ the reply handler cannot be called until the thread finished blocked
 +
 +[21:46:07] <​temporal_>​ (which is obvious :-) )
 +
 +[21:46:16] <​jtruelove_>​ right makes sense
 +
 +[21:46:35] <​jtruelove_>​ and i see in the event bus where you store the context of each registered handler for doing the callbacks
 +
 +[21:46:45] <​temporal_>​ aesteve if you consumer from a worker, then the message will be delivered by a worker thread
 +
 +[21:46:47] <​jtruelove_>​ on the specific eventloop
 +
 +[21:47:11] <​jtruelove_>​ in the case of the work thread you have a worker context i believe
 +
 +[21:47:23] <​temporal_>​ yes that context is the concurrency model
 +
 +[21:47:37] <​temporal_>​ of the message delivery
 +
 +[21:47:45] <​jtruelove_>​ yeah i basically assume it works the way i think it should and cross my fingers :)
 +
 +[21:48:05] <​jtruelove_>​ but it's always great to scratch the itch and look at the code and be like damn that is elegant
 +
 +[21:48:13] <​jtruelove_>​ i do love the architecture
 +
 +[21:48:18] <​temporal_>​ you should greet Tim Fox for that :-)
 +
 +[21:48:45] <​jtruelove_>​ i think i have via email before and with my support of the project :)
 +
 +[21:49:17] <​jtruelove_>​ it would be great to get design diagrams and overviews on the site to communicate some of this stuff out
 +
 +[21:49:43] <​temporal_>​ yes, if only we had more ppl contributing :)
 +
 +[21:50:03] <​jtruelove_>​ i do what i can to proselytize it
 +
 +[21:50:05] <​temporal_>​ the "​demystifying the event loop" article is an attempt for this
 +
 +[21:50:31] <​temporal_>​ no diagram, but a rather technical writing on how it works
 +
 +[21:50:52] <​jtruelove_>​ i may not have seen it, how netty handles the thread identifying is or isn't on the event loop so do it no or drop it in a queue is very clean
 +
 +[21:51:41] <​temporal_>​ that's netty yes
 +
 +[21:51:45] <​jtruelove_>​ it's very similar to an internal c# NIO framework i helped maintain many years ago but a lot more large scale
 +
 +[21:51:51] <​temporal_>​ that's soemthing they changed between version 3 and version 4
 +
 +[21:52:00] <​jtruelove_>​ and vertx more or less weds itself to the same event loop groups
 +
 +[21:52:17] <​temporal_>​ vertx reuses netty event loop system
 +
 +[21:52:34] <​temporal_>​ my opinion is that Netty is quite complex to program
 +
 +[21:52:44] <​jtruelove_>​ yeah i was looking at that in vertx impl and context code last week
 +
 +[21:52:44] <​temporal_>​ and definitely requires expertise
 +
 +[21:52:49] <​jtruelove_>​ agreed
 +
 +[21:53:03] <​temporal_>​ like doing a websocket handshake
 +
 +[21:53:20] <​temporal_>​ but I think that what netty is, and they don't need to do more
 +
 +[21:53:21] <​jtruelove_>​ when i left the company where i maintained that framework i found netty and was like damn this will do but it's a bitch to code in
 +
 +[21:53:45] <​temporal_>​ netty http2 is very diffferent from http
 +
 +[21:53:47] <​jtruelove_>​ i wanted to get out of low level serialization problems and things
 +
 +[21:54:02] <​temporal_>​ other examples
 +
 +[21:54:18] <​temporal_>​ netty provides flow control via channel readability and isWritable
 +
 +[21:54:31] <​temporal_>​ but there is no model for propagating back pressure
 +
 +[21:54:42] <​temporal_>​ vertx provides that for Netty and more (AsyncFile)
 +
 +[21:54:49] <​jtruelove_>​ so you can over queue?
 +
 +[21:55:01] <​temporal_>​ I'm talking of the Pump
 +
 +[21:55:17] <​jtruelove_>​ ah i haven'​t looked at that code much
 +
 +[21:55:36] <​temporal_>​ the pump use ReadStream / WriteStream
 +
 +[21:56:10] <​jtruelove_>​ from netty'​s NIO or just raw java nio?
 +
 +[21:56:47] <​temporal_>​ Netty NIO
 +
 +[21:56:55] <​temporal_>​ (that is java NIO under the hood)
 +
 +[21:57:06] <​temporal_>​ Netty NIO provides isWritable()
 +
 +[21:57:08] <​jtruelove_>​ yeah
 +
 +[21:57:13] <​temporal_>​ and can change channel readability
 +
 +[21:57:31] <​temporal_>​ for HTTP2 it's different because it's handled in the protocol
 +
 +[21:57:57] <​jtruelove_>​ interesting but transparent from the users thankfully
 +
 +[21:58:23] <​jtruelove_>​ i looked at manually doing http upgrades to a websocket to get back to your other reference, lots of weird magic
 +
 +[21:58:40] <​jtruelove_>​ back in vertx 2 as i don't think you could upgrade as easily as in 3
 +
 +[21:59:00] <​jtruelove_>​ the whole magic string business was really odd
 +
 +[21:59:12] <​jtruelove_>​ everyone has to send
 +
 +[22:01:50] <​jtruelove_>​ 258EAFA5-E914-47DA-95CA-C5AB0DC85B11
 +
 +[22:01:56] <​jtruelove_>​ at least the server
 +
 +[22:02:05] <​temporal_>​ you have one also in http2
 +
 +[22:02:31] <​temporal_>​ http://​blog.jgc.org/​2015/​11/​the-secret-message-hidden-in-every.html
 +
 +[22:02:40] <​temporal_>​ 0x505249202a20485454502f322e300d0a0d0a534d0d0a0d0a
 +
 +[22:02:52] <​temporal_>​ which is "PRI *
 +
 +[22:02:52] <​temporal_> ​   HTTP/​2.0\r\n\r\nSM\r\n\r\n"​
 +
 +[22:03:01] <​temporal_>​ (makes more sense)
 +
 +[22:03:08] <​temporal_>​ but it's called "​magic"​ :-)
 +
 +[22:03:13] <​jtruelove_>​ lol
 +
 +[22:03:49] <​jtruelove_>​ when i was first trying to figure out how to do the manual upgrade i thought the special key was for sure specific to my servers at least
 +
 +[22:03:54] <​jtruelove_>​ but nope it's global
 +
 +[22:22:04] <​jtruelove_>​ temporal_ one last question that was bugging me, looking at the timer code so when timers get scheduled on nettys event loop group do they run on a separate thread from that event loop group i/o? i know the callback is obviously run on the context of the verticle instance that scheduled it
 +
 +[22:22:39] <​jtruelove_>​ but if the timer runs on the same thread or gets picked up it seems to have a great chance of being imprecise depending on how much work got queued in front of it
 +
 +[22:23:03] <​jtruelove_>​ like if you sampled a heap for expired timers every millisecond or something
 +
 +[22:39:14] <​jtruelove_>​ actually it does work like i'm sorta describing
 +
 +[23:07:16] <​jtruelove_>​ so it can be imprecise but by design
 +
 +[23:43:54] *** ChanServ sets mode: +o temporalfox