Differences
This shows you the differences between two versions of the page.
— |
irc:1456182000 [2017/05/27 13:44] (current) |
||
---|---|---|---|
Line 1: | Line 1: | ||
+ | [10:42:14] <cescoffier> hi davsclaus | ||
+ | |||
+ | [10:42:16] <davsclaus> cescoffier okay got some code | ||
+ | |||
+ | [10:42:32] <davsclaus> will do a PR shortly with 50% done | ||
+ | |||
+ | [10:42:46] <cescoffier> cool, what are you improving ? | ||
+ | |||
+ | [10:43:03] <davsclaus> https://github.com/vert-x3/vertx-camel-bridge/pull/1 | ||
+ | |||
+ | [10:43:12] <davsclaus> to leverage camel's routing engine correctly | ||
+ | |||
+ | [10:43:21] <davsclaus> same kind of work needed in the outbound | ||
+ | |||
+ | [10:43:26] <davsclaus> will take a look in a while | ||
+ | |||
+ | [10:44:41] <cescoffier> so you use AsyncProcessor | ||
+ | |||
+ | [10:44:51] <cescoffier> in which thread is this processor called ? | ||
+ | |||
+ | [10:45:20] <davsclaus> the camel consumer | ||
+ | |||
+ | [10:45:23] <cescoffier> (same question for callback.done()) | ||
+ | |||
+ | [10:45:32] <cescoffier> so a thread pool managed by camel | ||
+ | |||
+ | [10:45:33] <davsclaus> but the reply part is the vertx thread | ||
+ | |||
+ | [10:45:55] <davsclaus> so if you do reply then you can "free" the consumer thread | ||
+ | |||
+ | [10:45:59] <davsclaus> when you return false | ||
+ | |||
+ | [10:46:25] <davsclaus> and leave it to vertx to resume the work when it has the reply | ||
+ | |||
+ | [10:46:50] <cescoffier> ok, cool | ||
+ | |||
+ | [10:47:08] <davsclaus> sadly AsyncProcessor is not a single interface so its not java 8 lambda readyu | ||
+ | |||
+ | [10:47:11] <cescoffier> looks much better than my attemp to check in the exchange was InOut | ||
+ | |||
+ | [10:47:42] <davsclaus> i will give a go on the outbound side as well and do a PR there | ||
+ | |||
+ | [10:47:56] <DP_2014> Hi, I have created a web app but as its getting more and more complex with hundreds or routes and handlers It becoming really messy, passing RoutingContext areound every wherem anyone know of a good structure to group and inherit handler functionality? | ||
+ | |||
+ | [10:49:21] <cescoffier> DP_2014 : the handler don't have to be in the same class, you can create one class per handler if you want | ||
+ | |||
+ | [10:50:32] <DP_2014> yes i have done that and grouped handlers in classes like controllers, but when i want base functionalitly and inherit the only way i can get around it is passing REouting context to each method | ||
+ | |||
+ | [10:50:52] <DP_2014> sorry passing RoutingContext | ||
+ | |||
+ | [10:52:05] <DP_2014> this is all pretty much just like controllers no other services I ahve about 20 other verticles that handle all the services, I am having trouble with the complexity of the routing | ||
+ | |||
+ | [10:55:36] <DP_2014> has anyone used routing with hundreds of routes? | ||
+ | |||
+ | [10:59:14] <cescoffier> ping pmlopes | ||
+ | |||
+ | [11:00:13] <cescoffier> pmlopes : DP_2014 has a question for you | ||
+ | |||
+ | [11:01:32] <DP_2014> Lol, I need extension methods like they have in C# | ||
+ | |||
+ | [11:01:56] <pmlopes> @DP_2014 the routing context is a dynamic object it represents the request so you cannot use it as a class variable and set it once on the contructor | ||
+ | |||
+ | [11:02:23] <DP_2014> yes i know, so i have to pass it to every method | ||
+ | |||
+ | [11:02:24] <cescoffier> davsclaus : I've merged your PR, re-eanbled the skipped test (as replies work now), and externalize the helper method into a Helper class. | ||
+ | |||
+ | [11:02:36] <davsclaus> cescoffier nice | ||
+ | |||
+ | [11:03:01] <cescoffier> davsclaus it starts to look pretty good (planning a write a blog post about it) | ||
+ | |||
+ | [11:03:02] <pmlopes> now depending on your objects you could use a generator like approach, you could have a generic class that on its handler method creates a new object of your classes and in that case it will have the routing context in the contructor | ||
+ | |||
+ | [11:03:17] <pmlopes> but this object cannot live longer than the request itself | ||
+ | |||
+ | [11:03:28] <DP_2014> yes thats what i have bee nthinking on | ||
+ | |||
+ | [11:10:26] <DP_2014> pmlopes do you see any downside in that approach? | ||
+ | |||
+ | [11:11:46] <DP_2014> I am also struggling with hundreds of route combination methods, with either a tonne of routes or a switch/case statment in the classes to direct to the correct method? Am i missing something obvious? | ||
+ | |||
+ | [11:12:41] <pmlopes> @DP_2014 if your objects do not do any long initialization, say for example that only set the RC to a object state, then I see them as a extended lambda so performance wise it should be negletable | ||
+ | |||
+ | [11:13:23] <pmlopes> @DP_2014 if you need switch cases to route to the right method, i guess that your API is not as simple as it should be, it shows that you have overlaping | ||
+ | |||
+ | [11:13:34] <pmlopes> maybe you should revisit that too | ||
+ | |||
+ | [11:14:00] <DP_2014> sorry I can either have hundreds of routes , or simple routes and then swtich statements | ||
+ | |||
+ | [11:14:10] <pmlopes> of course i'm saying this without looking at code | ||
+ | |||
+ | [11:14:45] <pmlopes> out of curiosity what kind of system are you building to require hundreds of routes? | ||
+ | |||
+ | [11:16:16] <DP_2014> Its a business system that is being ported for social collaborative scale | ||
+ | |||
+ | [11:17:39] <DP_2014> so full crm, financial accounting, manufacturing , hr, inventory, etc but on a collaborative model | ||
+ | |||
+ | [11:18:08] <pmlopes> so if you have simple routes and switch statement how would that look like? | ||
+ | |||
+ | [11:18:38] <DP_2014> it has been ported from c# | ||
+ | |||
+ | [11:18:49] <DP_2014> taken about a year to port | ||
+ | |||
+ | [11:19:35] <DP_2014> "//area1/action1 /area1/action2 ../areaN/actionN" | ||
+ | |||
+ | [11:20:14] <DP_2014> ite gets more complex than that | ||
+ | |||
+ | [11:20:33] <DP_2014> but router.route("/:vtxarea/:vtxaction") | ||
+ | |||
+ | [11:21:22] <pmlopes> so you could get a simple route with vars like "/:area/:action" and the handler would then create a object that would contain the RC and be suited to handle a specific area+context type | ||
+ | |||
+ | [11:21:34] <DP_2014> but its ususally within areas like router.route("/person/:vtxarea/:vtxaction").handler(programHandler); | ||
+ | |||
+ | [11:21:41] <DP_2014> but its ususally within areas like router.route("/person/:vtxarea/:vtxaction").handler(personHandler); | ||
+ | |||
+ | [11:23:29] <pmlopes> i guess you can hide the switch statements inside the constructor of your object or use some other smarter technique | ||
+ | |||
+ | [11:24:02] <pmlopes> like some hashing given the set of variables give me the object type that can handle it | ||
+ | |||
+ | [11:26:23] <DP_2014> or more clearly the url would be something like "/inventory/widget1/stock" would show stock levels for widget1 and "/inventory/widget1/purchasing" would show purchasing for widget1 handler would be inventoryHandler ie. router.route("/inventory/:vtxid/:vtxaction").handler(inventoryHandler); but in the inventory handler i have massive case statment for all the possible... | ||
+ | |||
+ | [11:26:24] <DP_2014> ...actions | ||
+ | |||
+ | [11:26:58] <DP_2014> now multipply this for tens of module areas, | ||
+ | |||
+ | [11:29:41] <pmlopes> well, you could try to handle this the other way around, so say you have the route: "/inventory/:vtxid/:vtxaction" you first add a handler that only looks at "vtaction" and sets the right object in the context "ctx.put("vtAction", new MyVTActionHandler(ctx, ctx.getParam("vtaction"))); and then ctx.next() | ||
+ | |||
+ | [11:30:04] <pmlopes> now on the following handler you always have the handler type resolved so no need to switch statements | ||
+ | |||
+ | [11:30:57] <pmlopes> you just need to do ctx.get("vtAction").runMyBusinessLogic(...) and what ever extra needed | ||
+ | |||
+ | [11:35:30] <DP_2014> .../stock action will call a method like a stock() that has completely different business logic than .../purchasing purchasing(). also if person puts url /inventory/widget1/badRequest it will still be picked up by the route but is invalid? I do chain the handlers now to session - > auth -> data handler ->db handler - >render/repsonse handler but in the... | ||
+ | |||
+ | [11:35:31] <DP_2014> ...middel the data and bus | ||
+ | |||
+ | [11:37:57] <pmlopes> you can validate if the action is valid, say having a list of valid actions and do something like: if (!actionsList.contains(ctx.getParam("vtaction"))) { ctx.fail(400); } else { ctx.next(); } | ||
+ | |||
+ | [11:38:12] <DP_2014> sorry, don't mean to be coming across as being argumentative, your advice has been great, especially the geneerator approach!! I am just trying to get my head around all the issues | ||
+ | |||
+ | [11:49:28] <davsclaus> cescoffier a PR with the outbound implemented | ||
+ | |||
+ | [11:49:33] <davsclaus> https://github.com/vert-x3/vertx-camel-bridge/pull/2 | ||
+ | |||
+ | [11:53:23] <cescoffier> thanks davsclaus, gonna have a look after lunch | ||
+ | |||
+ | [12:26:02] *** ChanServ sets mode: +o temporal_ | ||
+ | |||
+ | [13:22:18] <cescoffier> davsclaus : back, having a look | ||
+ | |||
+ | [13:36:49] <BadApe> i am using postgresql and the json field type, when i encode the result, the json object field gets converted to a string | ||
+ | |||
+ | [13:36:55] <BadApe> is there anyway around this? | ||
+ | |||
+ | [13:38:41] <cescoffier> BadApe: are you using the mysql/postgres client or the JDBC async client ? | ||
+ | |||
+ | [13:38:48] <BadApe> jdbc client | ||
+ | |||
+ | [13:39:27] <BadApe> MySQL / PostgreSQL client says tech preview | ||
+ | |||
+ | [13:39:40] <cescoffier> BadApe : I'm not sure we support the Json type. So it is probably passed as a String, and you would need to recereate the Json Object out of it | ||
+ | |||
+ | [13:39:57] <cescoffier> BadApe : yes the MySQL / Postgres client is a tech preview | ||
+ | |||
+ | [13:40:05] <BadApe> http://vertx.io/blog/vert-x3-and-postgresql-json-type/ :) | ||
+ | |||
+ | [13:40:20] <cescoffier> davsclaus : merged ! Thanks ! I've updated the documentation. So now it's time for the vert.x team review | ||
+ | |||
+ | [13:40:24] <BadApe> there is some support :) | ||
+ | |||
+ | [13:40:53] <cescoffier> BadApe : didn't read this article, I should ;-). | ||
+ | |||
+ | [13:40:56] <davsclaus> you may want to add a bit debug/trace logging on the points where camel and vertx "change hands" | ||
+ | |||
+ | [13:41:02] <davsclaus> it may help during some debugging etc | ||
+ | |||
+ | [13:42:04] <BadApe> i read all the blog posts, very enlightening for a newbie | ||
+ | |||
+ | [13:42:27] <BadApe> they should be changed into a beginners guide | ||
+ | |||
+ | [13:42:31] <cescoffier> BadApe : in the snippet, line 88: // here we return the document and note that i don't need to convert from JsonObject to String, PostgreSQL returns JSON as strings so less conversions, better performance! | ||
+ | |||
+ | [13:43:20] <cescoffier> davsclaus : yes, it would make sense to add a few log, I've done basic thread check to be sure we don't block the event loop | ||
+ | |||
+ | [13:43:36] <BadApe> i guess i will have to loop over the results | ||
+ | |||
+ | [13:44:21] <BadApe> it doesn't make using the json field type very useful for a rest interface | ||
+ | |||
+ | [13:45:06] <cescoffier> BadApe : as far as I understand, it's postgres that returns String objects | ||
+ | |||
+ | [13:45:12] <davsclaus> cescoffier +1 | ||
+ | |||
+ | [13:46:18] <BadApe> maybe i shouldn't use the json field type | ||
+ | |||
+ | [13:56:53] <BadApe> oh it was easier to manually covert it | ||
+ | |||
+ | [13:57:01] <BadApe> shame i don't know which are json fields | ||
+ | |||
+ | [17:26:51] <BadApe> so i don't really want to write a failure handler for every route, can i do something like http://pastebin.com/BZzf1FWM | ||
+ | |||
+ | [20:38:22] <jtruelove_> temporal_ has there been any thought on allowing AbstractVerticle to have an override like getVertxOptions where you could configure a custom SPI or something for instance | ||
+ | |||
+ | [20:38:41] <jtruelove_> i feel like i've asked this before but there was a chicken and egg problem with that | ||
+ | |||
+ | [22:41:31] <temporal_> jtruelove_ yes it would be a problem and it would also complexify, a verticle should not own a configuration for vertx | ||
+ | |||
+ | [22:41:41] <temporal_> what is the problem you are trying to solve ? | ||
+ | |||
+ | [22:52:34] *** ChanServ sets mode: +o temporalfox | ||
+ | |||
+ | [23:11:28] <jtruelove_> it would be nice to use say vertx-opentsdb and still use abstractverticle | ||
+ | |||
+ | [23:11:42] <jtruelove_> but you need to specify config VertxOptions in order to config the SPI | ||
+ | |||
+ | [23:16:12] <jtruelove_> temporal_ ^ | ||
+ | |||
+ | [23:27:07] <jtruelove_> err temporalfox ^ | ||
+ | |||
+ | [23:27:41] <temporalfox> jtruelove_ btw have you seen my mail about Thomas Segismont work on metrics ? | ||
+ | |||
+ | [23:28:48] <temporalfox> jtruelove_ what is your use case exactly ? | ||
+ | |||
+ | [23:28:52] <temporalfox> I mean how do you use it ? | ||
+ | |||
+ | [23:31:05] <jtruelove_> i saw a mail I have not fully pondered it, it sounded if I'm remembering correctly akin to some things that julien and i discussed a bit back | ||
+ | |||
+ | [23:31:30] <temporalfox> I'm julien :-) | ||
+ | |||
+ | [23:32:02] <jtruelove_> well then you :) never sure which is the real fox in here | ||
+ | |||
+ | [23:32:22] <jtruelove_> well we write most of our services using AbstractVerticle and a fatjar manifest that points at that implementor, doing that doesn't allow you to easily integrate with SPI | ||
+ | |||
+ | [23:32:45] <jtruelove_> unless you make a wrapper verticle for injecting our custom vertx-opentsdb options etc.. | ||
+ | |||
+ | [23:32:55] <temporalfox> you could make your own verticle factory perhaps | ||
+ | |||
+ | [23:33:06] <jtruelove_> so a while back when i did SPI we discussed it and hawkular and some of the similarities | ||
+ | |||
+ | [23:33:11] <temporalfox> that setup opentsdb options | ||
+ | |||
+ | [23:33:21] <temporalfox> based on the verticle class or annotations | ||
+ | |||
+ | [23:33:58] <temporalfox> actually : chicken egg problem | ||
+ | |||
+ | [23:34:24] <jtruelove_> yeah, you have to start the verticle with the vertxoptions | ||
+ | |||
+ | [23:34:38] <jtruelove_> it's how the SPI shit gets injected ultimately right | ||
+ | |||
+ | [23:34:41] <temporalfox> I think the best if you use a fat jar is to make your own Launcher | ||
+ | |||
+ | [23:34:48] <temporalfox> and extend the exisint CLI launcher | ||
+ | |||
+ | [23:34:58] <jtruelove_> yeah which will take the insatnces args and things and pass them in | ||
+ | |||
+ | [23:35:11] <jtruelove_> and just start the verticle | ||
+ | |||
+ | [23:35:11] <temporalfox> it will prepopulation the options | ||
+ | |||
+ | [23:35:13] <temporalfox> yes | ||
+ | |||
+ | [23:35:38] <jtruelove_> it would just be nice if you could have SPI options returned but i get why you can't | ||
+ | |||
+ | [23:36:04] <jtruelove_> on the thomas thing, i think you guys are discussing a common interface for reporting gauges, counters, and timers via the event bus | ||
+ | |||
+ | [23:36:07] <jtruelove_> and also maybe directly | ||
+ | |||
+ | [23:36:16] <temporalfox> we are discussing two things | ||
+ | |||
+ | [23:36:19] <jtruelove_> which is what vertx-opentsdb does more or less | ||
+ | |||
+ | [23:36:19] <temporalfox> 1/ what you said | ||
+ | |||
+ | [23:36:41] <temporalfox> 2/ abbstract the hawkular spi implementation to be decoupled from hawkular | ||
+ | |||
+ | [23:36:45] <temporalfox> and have hawkular only being a reporter | ||
+ | |||
+ | [23:37:11] <temporalfox> so the generic SPI impl gather statistics and periodically flush them in hawkular using a rest client | ||
+ | |||
+ | [23:37:20] <temporalfox> because these days many TSDB work this way | ||
+ | |||
+ | [23:37:30] <temporalfox> hawkular, opentsdb, influxdb, etc... | ||
+ | |||
+ | [23:37:35] <jtruelove_> yeah exactly | ||
+ | |||
+ | [23:37:42] <temporalfox> that would allow to have in common this collecting spi code | ||
+ | |||
+ | [23:37:43] <jtruelove_> some don't support gauges but other than that | ||
+ | |||
+ | [23:37:56] <jtruelove_> and be smart hopefully about batching and things | ||
+ | |||
+ | [23:37:59] <temporalfox> I think when you report you get generic statistics | ||
+ | |||
+ | [23:38:11] <temporalfox> and then it's up to the reporter to take care of this :- | ||
+ | |||
+ | [23:38:13] <temporalfox> :-) | ||
+ | |||
+ | [23:38:37] <temporalfox> I think thomas wants also to make a persistent queue | ||
+ | |||
+ | [23:38:47] <temporalfox> so if metric server is down | ||
+ | |||
+ | [23:38:51] <temporalfox> metrics are stored there | ||
+ | |||
+ | [23:38:54] <temporalfox> and queued | ||
+ | |||
+ | [23:38:57] <temporalfox> until it comes back | ||
+ | |||
+ | [23:39:36] <jtruelove_> interesting, could be good but could also be a pain in the ass | ||
+ | |||
+ | [23:39:52] <temporalfox> I think this would be optional | ||
+ | |||
+ | [23:39:52] <jtruelove_> depending on how long it would take data before purging | ||
+ | |||
+ | [23:40:11] <temporalfox> but you can discuss that on vertx-dev if you are concerned | ||
+ | |||
+ | [23:40:19] <jtruelove_> or popping longest in the queue data etc.. | ||
+ | |||
+ | [23:44:27] <jtruelove_> i'll try and engage on that thread | ||