Differences

This shows you the differences between two versions of the page.

Link to this comparison view

— irc:1456182000 [2017/05/27 13:44] (current)
Line 1: Line 1:
 +[10:42:14] <cescoffier> hi davsclaus
 +
 +[10:42:16] <davsclaus> cescoffier okay got some code
 +
 +[10:42:32] <davsclaus> will do a PR shortly with 50% done
 +
 +[10:42:46] <cescoffier> cool, what are you improving ?
 +
 +[10:43:03] <davsclaus> https://github.com/vert-x3/vertx-camel-bridge/pull/1
 +
 +[10:43:12] <davsclaus> to leverage camel's routing engine correctly
 +
 +[10:43:21] <davsclaus> same kind of work needed in the outbound
 +
 +[10:43:26] <davsclaus> will take a look in a while
 +
 +[10:44:41] <cescoffier> so you use AsyncProcessor
 +
 +[10:44:51] <cescoffier> in which thread is this processor called ?
 +
 +[10:45:20] <davsclaus> the camel consumer
 +
 +[10:45:23] <cescoffier> (same question for callback.done())
 +
 +[10:45:32] <cescoffier> so a thread pool managed by camel
 +
 +[10:45:33] <davsclaus> but the reply part is the vertx thread
 +
 +[10:45:55] <davsclaus> so if you do reply then you can "free" the consumer thread
 +
 +[10:45:59] <davsclaus> when you return false
 +
 +[10:46:25] <davsclaus> and leave it to vertx to resume the work when it has the reply
 +
 +[10:46:50] <cescoffier> ok, cool
 +
 +[10:47:08] <davsclaus> sadly AsyncProcessor is not a single interface so its not java 8 lambda readyu
 +
 +[10:47:11] <cescoffier> looks much better than my attemp to check in the exchange was InOut
 +
 +[10:47:42] <davsclaus> i will give a go on the outbound side as well and do a PR there
 +
 +[10:47:56] <DP_2014> Hi,  I have  created a web  app but as  its getting more and more complex with  hundreds or routes   and handlers It becoming really messy,  passing RoutingContext areound every wherem  anyone know of a good structure  to group and inherit handler functionality?
 +
 +[10:49:21] <cescoffier> DP_2014 : the handler don't have to be in the same class, you can create one class per handler if you want
 +
 +[10:50:32] <DP_2014> yes i have done that and grouped handlers in classes like controllers,  but when i want  base functionalitly and  inherit the only way i can get  around it is passing REouting context to each method
 +
 +[10:50:52] <DP_2014> sorry passing RoutingContext
 +
 +[10:52:05] <DP_2014> this is all pretty much  just like  controllers no other services I ahve about 20 other verticles that handle all the services, I am having trouble with the complexity of the routing
 +
 +[10:55:36] <DP_2014> has anyone used routing with hundreds of routes?
 +
 +[10:59:14] <cescoffier> ping pmlopes
 +
 +[11:00:13] <cescoffier> pmlopes : DP_2014 has a question for you
 +
 +[11:01:32] <DP_2014> Lol,     I need extension methods like they have in C#
 +
 +[11:01:56] <pmlopes> @DP_2014 the routing context is a dynamic object it represents the request so you cannot use it as a class variable and set it once on the contructor
 +
 +[11:02:23] <DP_2014> yes i know,   so i have to pass it to every method
 +
 +[11:02:24] <cescoffier> davsclaus : I've merged your PR, re-eanbled the skipped test (as replies work now), and externalize the helper method into a Helper class.
 +
 +[11:02:36] <davsclaus> cescoffier  nice
 +
 +[11:03:01] <cescoffier> davsclaus it starts to look pretty good (planning a write a blog post about it)
 +
 +[11:03:02] <pmlopes> now depending on your objects you could use a generator like approach, you could have a generic class that on its handler method creates a new object of your classes and in that case it will have the routing context in the contructor
 +
 +[11:03:17] <pmlopes> but this object cannot live longer than the request itself
 +
 +[11:03:28] <DP_2014> yes thats what i have bee nthinking on
 +
 +[11:10:26] <DP_2014> pmlopes do you  see any downside in that approach?
 +
 +[11:11:46] <DP_2014> I am also struggling with hundreds of route combination methods, with either a tonne of routes or a switch/case statment in the classes to direct to the correct method?  Am i missing  something obvious?
 +
 +[11:12:41] <pmlopes> @DP_2014 if your objects do not do any long initialization, say for example that only set the RC to a object state, then I see them as a extended lambda so performance wise it should be negletable
 +
 +[11:13:23] <pmlopes> @DP_2014 if you need switch cases to route to the right method, i guess that your API is not as simple as it should be, it shows that you have overlaping
 +
 +[11:13:34] <pmlopes> maybe you should revisit that too
 +
 +[11:14:00] <DP_2014> sorry   I can either have  hundreds of routes ,   or simple routes   and then swtich statements
 +
 +[11:14:10] <pmlopes> of course i'm saying this without looking at code
 +
 +[11:14:45] <pmlopes> out of curiosity what kind of system are you building to require hundreds of routes?
 +
 +[11:16:16] <DP_2014> Its a business system that is being ported  for social collaborative scale
 +
 +[11:17:39] <DP_2014> so full crm, financial accounting, manufacturing , hr, inventory, etc  but on a collaborative model
 +
 +[11:18:08] <pmlopes> so if you have simple routes and switch statement how would that look like?
 +
 +[11:18:38] <DP_2014> it has been ported from c#
 +
 +[11:18:49] <DP_2014> taken about a year to port
 +
 +[11:19:35] <DP_2014> "//area1/action1 /area1/action2    ../areaN/actionN"
 +
 +[11:20:14] <DP_2014> ite gets more complex than that
 +
 +[11:20:33] <DP_2014> but router.route("/:vtxarea/:vtxaction")
 +
 +[11:21:22] <pmlopes> so you could get a simple route with vars like "/:area/:action" and the handler would then create a object that would contain the RC and be suited to handle a specific area+context type
 +
 +[11:21:34] <DP_2014> but its ususally within areas like   router.route("/person/:vtxarea/:vtxaction").handler(programHandler);
 +
 +[11:21:41] <DP_2014> but its ususally within areas like   router.route("/person/:vtxarea/:vtxaction").handler(personHandler);
 +
 +[11:23:29] <pmlopes> i guess you can hide the switch statements inside the constructor of your object or use some other smarter technique
 +
 +[11:24:02] <pmlopes> like some hashing given the set of variables give me the object type that can handle it
 +
 +[11:26:23] <DP_2014> or more clearly the  url would be something like     "/inventory/widget1/stock"        would   show stock levels for widget1  and   "/inventory/widget1/purchasing"     would show purchasing for widget1  handler would be inventoryHandler  ie.   router.route("/inventory/:vtxid/:vtxaction").handler(inventoryHandler); but in the inventory handler i have massive case statment for all the possible...
 +
 +[11:26:24] <DP_2014> ...actions
 +
 +[11:26:58] <DP_2014> now multipply this for tens of module areas,
 +
 +[11:29:41] <pmlopes> well, you could try to handle this the other way around, so say you have the route: "/inventory/:vtxid/:vtxaction" you first add a handler that only looks at "vtaction" and sets the right object in the context "ctx.put("vtAction", new MyVTActionHandler(ctx, ctx.getParam("vtaction"))); and then ctx.next()
 +
 +[11:30:04] <pmlopes> now on the following handler you always have the handler type resolved so no need to switch statements
 +
 +[11:30:57] <pmlopes> you just  need to do ctx.get("vtAction").runMyBusinessLogic(...) and what ever extra needed
 +
 +[11:35:30] <DP_2014>  .../stock action will call  a method like a stock() that has completely different business  logic than  .../purchasing   purchasing().   also   if person puts  url   /inventory/widget1/badRequest      it will still be picked up by the route but is invalid?         I do chain the handlers now  to    session - >  auth -> data handler ->db handler - >render/repsonse handler          but in the...
 +
 +[11:35:31] <DP_2014> ...middel the data and bus
 +
 +[11:37:57] <pmlopes> you can validate if the action is valid, say having a list of valid actions and do something like: if (!actionsList.contains(ctx.getParam("vtaction"))) { ctx.fail(400); } else { ctx.next(); }
 +
 +[11:38:12] <DP_2014> sorry, don't mean to be coming across as being argumentative, your advice  has been great,  especially the   geneerator approach!!    I am just trying to get my head around  all the issues
 +
 +[11:49:28] <davsclaus> cescoffier a PR with the outbound implemented
 +
 +[11:49:33] <davsclaus> https://github.com/vert-x3/vertx-camel-bridge/pull/2
 +
 +[11:53:23] <cescoffier> thanks davsclaus, gonna have a look after lunch
 +
 +[12:26:02] *** ChanServ sets mode: +o temporal_
 +
 +[13:22:18] <cescoffier> davsclaus : back, having a look
 +
 +[13:36:49] <BadApe> i am using postgresql and the json field type, when i encode the result, the json object field gets converted to a string
 +
 +[13:36:55] <BadApe> is there anyway around this?
 +
 +[13:38:41] <cescoffier> BadApe: are you using the mysql/postgres client or the JDBC async client ?
 +
 +[13:38:48] <BadApe> jdbc client
 +
 +[13:39:27] <BadApe> MySQL / PostgreSQL client says tech preview
 +
 +[13:39:40] <cescoffier> BadApe : I'm not sure we support the Json type. So it is probably passed as a String, and you would need to recereate the Json Object out of it
 +
 +[13:39:57] <cescoffier> BadApe : yes the MySQL / Postgres client is a tech preview
 +
 +[13:40:05] <BadApe> http://vertx.io/blog/vert-x3-and-postgresql-json-type/ :)
 +
 +[13:40:20] <cescoffier> davsclaus : merged ! Thanks ! I've updated the documentation. So now it's time for the vert.x team review
 +
 +[13:40:24] <BadApe> there is some support :)
 +
 +[13:40:53] <cescoffier> BadApe : didn't read this article, I should ;-).
 +
 +[13:40:56] <davsclaus> you may want to add a bit debug/trace logging on the points where camel and vertx "change hands"
 +
 +[13:41:02] <davsclaus> it may help during some debugging etc
 +
 +[13:42:04] <BadApe> i read all the blog posts, very enlightening for a newbie
 +
 +[13:42:27] <BadApe> they should be changed into a beginners guide
 +
 +[13:42:31] <cescoffier> BadApe : in the snippet, line 88:  // here we return the document and note that i don't need to convert from JsonObject to String, PostgreSQL returns JSON as strings so less conversions, better performance!
 +
 +[13:43:20] <cescoffier> davsclaus : yes, it would make sense to add a few log, I've done basic thread check to be sure we don't block the event loop
 +
 +[13:43:36] <BadApe> i guess i will have to loop over the results
 +
 +[13:44:21] <BadApe> it doesn't make using the json field type very useful for a rest interface
 +
 +[13:45:06] <cescoffier> BadApe : as far as I understand, it's postgres that returns String objects
 +
 +[13:45:12] <davsclaus> cescoffier +1
 +
 +[13:46:18] <BadApe> maybe i shouldn't use the json field type
 +
 +[13:56:53] <BadApe> oh it was easier to manually covert it
 +
 +[13:57:01] <BadApe> shame i don't know which are json fields
 +
 +[17:26:51] <BadApe> so i don't really want to write a failure handler for every route, can i do something like http://pastebin.com/BZzf1FWM
 +
 +[20:38:22] <jtruelove_> temporal_ has there been any thought on allowing AbstractVerticle to have an override like getVertxOptions where you could configure a custom SPI or something for instance
 +
 +[20:38:41] <jtruelove_> i feel like i've asked this before but there was a chicken and egg problem with that
 +
 +[22:41:31] <temporal_> jtruelove_ yes it would be a problem and it would also complexify, a verticle should not own a configuration for vertx
 +
 +[22:41:41] <temporal_> what is the problem you are trying to solve ?
 +
 +[22:52:34] *** ChanServ sets mode: +o temporalfox
 +
 +[23:11:28] <jtruelove_> it would be nice to use say vertx-opentsdb and still use abstractverticle
 +
 +[23:11:42] <jtruelove_> but you need to specify config VertxOptions in order to config the SPI
 +
 +[23:16:12] <jtruelove_> temporal_ ^
 +
 +[23:27:07] <jtruelove_> err temporalfox ^
 +
 +[23:27:41] <temporalfox> jtruelove_ btw have you seen my mail about Thomas Segismont work on metrics ?
 +
 +[23:28:48] <temporalfox> jtruelove_ what is your use case exactly ?
 +
 +[23:28:52] <temporalfox> I mean how do you use it ?
 +
 +[23:31:05] <jtruelove_> i saw a mail I have not fully pondered it, it sounded if I'm remembering correctly akin to some things that julien and i discussed a bit back
 +
 +[23:31:30] <temporalfox> I'm julien :-)
 +
 +[23:32:02] <jtruelove_> well then you :) never sure which is the real fox in here
 +
 +[23:32:22] <jtruelove_> well we write most of our services using AbstractVerticle and a fatjar manifest that points at that implementor, doing that doesn't allow you to easily integrate with SPI
 +
 +[23:32:45] <jtruelove_> unless you make a wrapper verticle for injecting our custom vertx-opentsdb options etc..
 +
 +[23:32:55] <temporalfox> you could make your own verticle factory perhaps
 +
 +[23:33:06] <jtruelove_> so a while back when i did SPI we discussed it and hawkular and some of the similarities
 +
 +[23:33:11] <temporalfox> that setup opentsdb options
 +
 +[23:33:21] <temporalfox> based on the verticle class or annotations
 +
 +[23:33:58] <temporalfox> actually : chicken egg problem
 +
 +[23:34:24] <jtruelove_> yeah, you have to start the verticle with the vertxoptions
 +
 +[23:34:38] <jtruelove_> it's how the SPI shit gets injected ultimately right
 +
 +[23:34:41] <temporalfox> I think the best if you use a fat jar is to make your own Launcher
 +
 +[23:34:48] <temporalfox> and extend the exisint CLI launcher
 +
 +[23:34:58] <jtruelove_> yeah which will take the insatnces args and things and pass them in
 +
 +[23:35:11] <jtruelove_> and just start the verticle
 +
 +[23:35:11] <temporalfox> it will prepopulation the options
 +
 +[23:35:13] <temporalfox> yes
 +
 +[23:35:38] <jtruelove_> it would just be nice if you could have SPI options returned but i get why you can't
 +
 +[23:36:04] <jtruelove_> on the thomas thing, i think you guys are discussing a common interface for reporting gauges, counters, and timers via the event bus
 +
 +[23:36:07] <jtruelove_> and also maybe directly
 +
 +[23:36:16] <temporalfox> we are discussing two things
 +
 +[23:36:19] <jtruelove_> which is what vertx-opentsdb does more or less
 +
 +[23:36:19] <temporalfox> 1/ what you said
 +
 +[23:36:41] <temporalfox> 2/ abbstract the hawkular spi implementation to be decoupled from hawkular
 +
 +[23:36:45] <temporalfox> and have hawkular only being a reporter
 +
 +[23:37:11] <temporalfox> so the generic SPI impl gather statistics and periodically flush them in hawkular using a rest client
 +
 +[23:37:20] <temporalfox> because these days many TSDB work this way
 +
 +[23:37:30] <temporalfox> hawkular, opentsdb, influxdb, etc...
 +
 +[23:37:35] <jtruelove_> yeah exactly
 +
 +[23:37:42] <temporalfox> that would allow to have in common this collecting spi code
 +
 +[23:37:43] <jtruelove_> some don't support gauges but other than that
 +
 +[23:37:56] <jtruelove_> and be smart hopefully about batching and things
 +
 +[23:37:59] <temporalfox> I think when you report you get generic statistics
 +
 +[23:38:11] <temporalfox> and then it's up to the reporter to take care of this :-
 +
 +[23:38:13] <temporalfox> :-)
 +
 +[23:38:37] <temporalfox> I think thomas wants also to make a persistent queue
 +
 +[23:38:47] <temporalfox> so if metric server is down
 +
 +[23:38:51] <temporalfox> metrics are stored there
 +
 +[23:38:54] <temporalfox> and queued
 +
 +[23:38:57] <temporalfox> until it comes back
 +
 +[23:39:36] <jtruelove_> interesting, could be good but could also be a pain in the ass
 +
 +[23:39:52] <temporalfox> I think this would be optional
 +
 +[23:39:52] <jtruelove_> depending on how long it would take data before purging
 +
 +[23:40:11] <temporalfox> but you can discuss that on vertx-dev if you are concerned
 +
 +[23:40:19] <jtruelove_> or popping longest in the queue data etc..
 +
 +[23:44:27] <jtruelove_> i'll try and engage on that thread