Differences

This shows you the differences between two versions of the page.

Link to this comparison view

irc:1456182000 [2017/05/27 13:44] (current)
Line 1: Line 1:
 +[10:42:14] <​cescoffier>​ hi davsclaus
 +
 +[10:42:16] <​davsclaus>​ cescoffier okay got some code
 +
 +[10:42:32] <​davsclaus>​ will do a PR shortly with 50% done
 +
 +[10:42:46] <​cescoffier>​ cool, what are you improving ?
 +
 +[10:43:03] <​davsclaus>​ https://​github.com/​vert-x3/​vertx-camel-bridge/​pull/​1
 +
 +[10:43:12] <​davsclaus>​ to leverage camel'​s routing engine correctly
 +
 +[10:43:21] <​davsclaus>​ same kind of work needed in the outbound
 +
 +[10:43:26] <​davsclaus>​ will take a look in a while
 +
 +[10:44:41] <​cescoffier>​ so you use AsyncProcessor
 +
 +[10:44:51] <​cescoffier>​ in which thread is this processor called ?
 +
 +[10:45:20] <​davsclaus>​ the camel consumer
 +
 +[10:45:23] <​cescoffier>​ (same question for callback.done())
 +
 +[10:45:32] <​cescoffier>​ so a thread pool managed by camel
 +
 +[10:45:33] <​davsclaus>​ but the reply part is the vertx thread
 +
 +[10:45:55] <​davsclaus>​ so if you do reply then you can "​free"​ the consumer thread
 +
 +[10:45:59] <​davsclaus>​ when you return false
 +
 +[10:46:25] <​davsclaus>​ and leave it to vertx to resume the work when it has the reply
 +
 +[10:46:50] <​cescoffier>​ ok, cool
 +
 +[10:47:08] <​davsclaus>​ sadly AsyncProcessor is not a single interface so its not java 8 lambda readyu
 +
 +[10:47:11] <​cescoffier>​ looks much better than my attemp to check in the exchange was InOut
 +
 +[10:47:42] <​davsclaus>​ i will give a go on the outbound side as well and do a PR there
 +
 +[10:47:56] <​DP_2014>​ Hi,  I have  created a web  app but as  its getting more and more complex with  hundreds or routes ​  and handlers It becoming really messy, ​ passing RoutingContext areound every wherem ​ anyone know of a good structure ​ to group and inherit handler functionality?​
 +
 +[10:49:21] <​cescoffier>​ DP_2014 : the handler don't have to be in the same class, you can create one class per handler if you want
 +
 +[10:50:32] <​DP_2014>​ yes i have done that and grouped handlers in classes like controllers, ​ but when i want  base functionalitly and  inherit the only way i can get  around it is passing REouting context to each method
 +
 +[10:50:52] <​DP_2014>​ sorry passing RoutingContext
 +
 +[10:52:05] <​DP_2014>​ this is all pretty much  just like  controllers no other services I ahve about 20 other verticles that handle all the services, I am having trouble with the complexity of the routing
 +
 +[10:55:36] <​DP_2014>​ has anyone used routing with hundreds of routes?
 +
 +[10:59:14] <​cescoffier>​ ping pmlopes
 +
 +[11:00:13] <​cescoffier>​ pmlopes : DP_2014 has a question for you
 +
 +[11:01:32] <​DP_2014>​ Lol,     I need extension methods like they have in C#
 +
 +[11:01:56] <​pmlopes>​ @DP_2014 the routing context is a dynamic object it represents the request so you cannot use it as a class variable and set it once on the contructor
 +
 +[11:02:23] <​DP_2014>​ yes i know,   so i have to pass it to every method
 +
 +[11:02:24] <​cescoffier>​ davsclaus : I've merged your PR, re-eanbled the skipped test (as replies work now), and externalize the helper method into a Helper class.
 +
 +[11:02:36] <​davsclaus>​ cescoffier ​ nice
 +
 +[11:03:01] <​cescoffier>​ davsclaus it starts to look pretty good (planning a write a blog post about it)
 +
 +[11:03:02] <​pmlopes>​ now depending on your objects you could use a generator like approach, you could have a generic class that on its handler method creates a new object of your classes and in that case it will have the routing context in the contructor
 +
 +[11:03:17] <​pmlopes>​ but this object cannot live longer than the request itself
 +
 +[11:03:28] <​DP_2014>​ yes thats what i have bee nthinking on
 +
 +[11:10:26] <​DP_2014>​ pmlopes do you  see any downside in that approach?
 +
 +[11:11:46] <​DP_2014>​ I am also struggling with hundreds of route combination methods, with either a tonne of routes or a switch/case statment in the classes to direct to the correct method? ​ Am i missing ​ something obvious?
 +
 +[11:12:41] <​pmlopes>​ @DP_2014 if your objects do not do any long initialization,​ say for example that only set the RC to a object state, then I see them as a extended lambda so performance wise it should be negletable
 +
 +[11:13:23] <​pmlopes>​ @DP_2014 if you need switch cases to route to the right method, i guess that your API is not as simple as it should be, it shows that you have overlaping
 +
 +[11:13:34] <​pmlopes>​ maybe you should revisit that too
 +
 +[11:14:00] <​DP_2014>​ sorry   I can either have  hundreds of routes ,   or simple routes ​  and then swtich statements
 +
 +[11:14:10] <​pmlopes>​ of course i'm saying this without looking at code
 +
 +[11:14:45] <​pmlopes>​ out of curiosity what kind of system are you building to require hundreds of routes?
 +
 +[11:16:16] <​DP_2014>​ Its a business system that is being ported ​ for social collaborative scale
 +
 +[11:17:39] <​DP_2014>​ so full crm, financial accounting, manufacturing , hr, inventory, etc  but on a collaborative model
 +
 +[11:18:08] <​pmlopes>​ so if you have simple routes and switch statement how would that look like?
 +
 +[11:18:38] <​DP_2014>​ it has been ported from c#
 +
 +[11:18:49] <​DP_2014>​ taken about a year to port
 +
 +[11:19:35] <​DP_2014>​ "//​area1/​action1 /​area1/​action2 ​   ../​areaN/​actionN"​
 +
 +[11:20:14] <​DP_2014>​ ite gets more complex than that
 +
 +[11:20:33] <​DP_2014>​ but router.route("/:​vtxarea/:​vtxaction"​)
 +
 +[11:21:22] <​pmlopes>​ so you could get a simple route with vars like "/:​area/:​action"​ and the handler would then create a object that would contain the RC and be suited to handle a specific area+context type
 +
 +[11:21:34] <​DP_2014>​ but its ususally within areas like   ​router.route("/​person/:​vtxarea/:​vtxaction"​).handler(programHandler);​
 +
 +[11:21:41] <​DP_2014>​ but its ususally within areas like   ​router.route("/​person/:​vtxarea/:​vtxaction"​).handler(personHandler);​
 +
 +[11:23:29] <​pmlopes>​ i guess you can hide the switch statements inside the constructor of your object or use some other smarter technique
 +
 +[11:24:02] <​pmlopes>​ like some hashing given the set of variables give me the object type that can handle it
 +
 +[11:26:23] <​DP_2014>​ or more clearly the  url would be something like     "/​inventory/​widget1/​stock" ​       would   show stock levels for widget1 ​ and   "/​inventory/​widget1/​purchasing" ​    would show purchasing for widget1 ​ handler would be inventoryHandler ​ ie.   ​router.route("/​inventory/:​vtxid/:​vtxaction"​).handler(inventoryHandler);​ but in the inventory handler i have massive case statment for all the possible...
 +
 +[11:26:24] <​DP_2014>​ ...actions
 +
 +[11:26:58] <​DP_2014>​ now multipply this for tens of module areas,
 +
 +[11:29:41] <​pmlopes>​ well, you could try to handle this the other way around, so say you have the route: "/​inventory/:​vtxid/:​vtxaction"​ you first add a handler that only looks at "​vtaction"​ and sets the right object in the context "​ctx.put("​vtAction",​ new MyVTActionHandler(ctx,​ ctx.getParam("​vtaction"​)));​ and then ctx.next()
 +
 +[11:30:04] <​pmlopes>​ now on the following handler you always have the handler type resolved so no need to switch statements
 +
 +[11:30:57] <​pmlopes>​ you just  need to do ctx.get("​vtAction"​).runMyBusinessLogic(...) and what ever extra needed
 +
 +[11:35:30] <​DP_2014> ​ .../stock action will call  a method like a stock() that has completely different business ​ logic than  .../​purchasing ​  ​purchasing(). ​  ​also ​  if person puts  url   /​inventory/​widget1/​badRequest ​     it will still be picked up by the route but is invalid? ​        I do chain the handlers now  to    session - >  auth -> data handler ->db handler - >​render/​repsonse handler ​         but in the...
 +
 +[11:35:31] <​DP_2014>​ ...middel the data and bus
 +
 +[11:37:57] <​pmlopes>​ you can validate if the action is valid, say having a list of valid actions and do something like: if (!actionsList.contains(ctx.getParam("​vtaction"​))) { ctx.fail(400);​ } else { ctx.next(); }
 +
 +[11:38:12] <​DP_2014>​ sorry, don't mean to be coming across as being argumentative,​ your advice ​ has been great, ​ especially the   ​geneerator approach!! ​   I am just trying to get my head around ​ all the issues
 +
 +[11:49:28] <​davsclaus>​ cescoffier a PR with the outbound implemented
 +
 +[11:49:33] <​davsclaus>​ https://​github.com/​vert-x3/​vertx-camel-bridge/​pull/​2
 +
 +[11:53:23] <​cescoffier>​ thanks davsclaus, gonna have a look after lunch
 +
 +[12:26:02] *** ChanServ sets mode: +o temporal_
 +
 +[13:22:18] <​cescoffier>​ davsclaus : back, having a look
 +
 +[13:36:49] <​BadApe>​ i am using postgresql and the json field type, when i encode the result, the json object field gets converted to a string
 +
 +[13:36:55] <​BadApe>​ is there anyway around this?
 +
 +[13:38:41] <​cescoffier>​ BadApe: are you using the mysql/​postgres client or the JDBC async client ?
 +
 +[13:38:48] <​BadApe>​ jdbc client
 +
 +[13:39:27] <​BadApe>​ MySQL / PostgreSQL client says tech preview
 +
 +[13:39:40] <​cescoffier>​ BadApe : I'm not sure we support the Json type. So it is probably passed as a String, and you would need to recereate the Json Object out of it
 +
 +[13:39:57] <​cescoffier>​ BadApe : yes the MySQL / Postgres client is a tech preview
 +
 +[13:40:05] <​BadApe>​ http://​vertx.io/​blog/​vert-x3-and-postgresql-json-type/​ :)
 +
 +[13:40:20] <​cescoffier>​ davsclaus : merged ! Thanks ! I've updated the documentation. So now it's time for the vert.x team review
 +
 +[13:40:24] <​BadApe>​ there is some support :)
 +
 +[13:40:53] <​cescoffier>​ BadApe : didn't read this article, I should ;-).
 +
 +[13:40:56] <​davsclaus>​ you may want to add a bit debug/trace logging on the points where camel and vertx "​change hands"
 +
 +[13:41:02] <​davsclaus>​ it may help during some debugging etc
 +
 +[13:42:04] <​BadApe>​ i read all the blog posts, very enlightening for a newbie
 +
 +[13:42:27] <​BadApe>​ they should be changed into a beginners guide
 +
 +[13:42:31] <​cescoffier>​ BadApe : in the snippet, line 88:  // here we return the document and note that i don't need to convert from JsonObject to String, PostgreSQL returns JSON as strings so less conversions,​ better performance!
 +
 +[13:43:20] <​cescoffier>​ davsclaus : yes, it would make sense to add a few log, I've done basic thread check to be sure we don't block the event loop
 +
 +[13:43:36] <​BadApe>​ i guess i will have to loop over the results
 +
 +[13:44:21] <​BadApe>​ it doesn'​t make using the json field type very useful for a rest interface
 +
 +[13:45:06] <​cescoffier>​ BadApe : as far as I understand, it's postgres that returns String objects
 +
 +[13:45:12] <​davsclaus>​ cescoffier +1
 +
 +[13:46:18] <​BadApe>​ maybe i shouldn'​t use the json field type
 +
 +[13:56:53] <​BadApe>​ oh it was easier to manually covert it
 +
 +[13:57:01] <​BadApe>​ shame i don't know which are json fields
 +
 +[17:26:51] <​BadApe>​ so i don't really want to write a failure handler for every route, can i do something like http://​pastebin.com/​BZzf1FWM
 +
 +[20:38:22] <​jtruelove_>​ temporal_ has there been any thought on allowing AbstractVerticle to have an override like getVertxOptions where you could configure a custom SPI or something for instance
 +
 +[20:38:41] <​jtruelove_>​ i feel like i've asked this before but there was a chicken and egg problem with that
 +
 +[22:41:31] <​temporal_>​ jtruelove_ yes it would be a problem and it would also complexify, a verticle should not own a configuration for vertx
 +
 +[22:41:41] <​temporal_>​ what is the problem you are trying to solve ?
 +
 +[22:52:34] *** ChanServ sets mode: +o temporalfox
 +
 +[23:11:28] <​jtruelove_>​ it would be nice to use say vertx-opentsdb and still use abstractverticle
 +
 +[23:11:42] <​jtruelove_>​ but you need to specify config VertxOptions in order to config the SPI
 +
 +[23:16:12] <​jtruelove_>​ temporal_ ^
 +
 +[23:27:07] <​jtruelove_>​ err temporalfox ^
 +
 +[23:27:41] <​temporalfox>​ jtruelove_ btw have you seen my mail about Thomas Segismont work on metrics ?
 +
 +[23:28:48] <​temporalfox>​ jtruelove_ what is your use case exactly ?
 +
 +[23:28:52] <​temporalfox>​ I mean how do you use it ?
 +
 +[23:31:05] <​jtruelove_>​ i saw a mail I have not fully pondered it, it sounded if I'm remembering correctly akin to some things that julien and i discussed a bit back
 +
 +[23:31:30] <​temporalfox>​ I'm julien :-)
 +
 +[23:32:02] <​jtruelove_>​ well then you :) never sure which is the real fox in here
 +
 +[23:32:22] <​jtruelove_>​ well we write most of our services using AbstractVerticle and a fatjar manifest that points at that implementor,​ doing that doesn'​t allow you to easily integrate with SPI
 +
 +[23:32:45] <​jtruelove_>​ unless you make a wrapper verticle for injecting our custom vertx-opentsdb options etc..
 +
 +[23:32:55] <​temporalfox>​ you could make your own verticle factory perhaps
 +
 +[23:33:06] <​jtruelove_>​ so a while back when i did SPI we discussed it and hawkular and some of the similarities
 +
 +[23:33:11] <​temporalfox>​ that setup opentsdb options
 +
 +[23:33:21] <​temporalfox>​ based on the verticle class or annotations
 +
 +[23:33:58] <​temporalfox>​ actually : chicken egg problem
 +
 +[23:34:24] <​jtruelove_>​ yeah, you have to start the verticle with the vertxoptions
 +
 +[23:34:38] <​jtruelove_>​ it's how the SPI shit gets injected ultimately right
 +
 +[23:34:41] <​temporalfox>​ I think the best if you use a fat jar is to make your own Launcher
 +
 +[23:34:48] <​temporalfox>​ and extend the exisint CLI launcher
 +
 +[23:34:58] <​jtruelove_>​ yeah which will take the insatnces args and things and pass them in
 +
 +[23:35:11] <​jtruelove_>​ and just start the verticle
 +
 +[23:35:11] <​temporalfox>​ it will prepopulation the options
 +
 +[23:35:13] <​temporalfox>​ yes
 +
 +[23:35:38] <​jtruelove_>​ it would just be nice if you could have SPI options returned but i get why you can't
 +
 +[23:36:04] <​jtruelove_>​ on the thomas thing, i think you guys are discussing a common interface for reporting gauges, counters, and timers via the event bus
 +
 +[23:36:07] <​jtruelove_>​ and also maybe directly
 +
 +[23:36:16] <​temporalfox>​ we are discussing two things
 +
 +[23:36:19] <​jtruelove_>​ which is what vertx-opentsdb does more or less
 +
 +[23:36:19] <​temporalfox>​ 1/ what you said
 +
 +[23:36:41] <​temporalfox>​ 2/ abbstract the hawkular spi implementation to be decoupled from hawkular
 +
 +[23:36:45] <​temporalfox>​ and have hawkular only being a reporter
 +
 +[23:37:11] <​temporalfox>​ so the generic SPI impl gather statistics and periodically flush them in hawkular using a rest client
 +
 +[23:37:20] <​temporalfox>​ because these days many TSDB work this way
 +
 +[23:37:30] <​temporalfox>​ hawkular, opentsdb, influxdb, etc...
 +
 +[23:37:35] <​jtruelove_>​ yeah exactly
 +
 +[23:37:42] <​temporalfox>​ that would allow to have in common this collecting spi code
 +
 +[23:37:43] <​jtruelove_>​ some don't support gauges but other than that
 +
 +[23:37:56] <​jtruelove_>​ and be smart hopefully about batching and things
 +
 +[23:37:59] <​temporalfox>​ I think when you report you get generic statistics
 +
 +[23:38:11] <​temporalfox>​ and then it's up to the reporter to take care of this :-
 +
 +[23:38:13] <​temporalfox>​ :-)
 +
 +[23:38:37] <​temporalfox>​ I think thomas wants also to make a persistent queue
 +
 +[23:38:47] <​temporalfox>​ so if metric server is down
 +
 +[23:38:51] <​temporalfox>​ metrics are stored there
 +
 +[23:38:54] <​temporalfox>​ and queued
 +
 +[23:38:57] <​temporalfox>​ until it comes back
 +
 +[23:39:36] <​jtruelove_>​ interesting,​ could be good but could also be a pain in the ass
 +
 +[23:39:52] <​temporalfox>​ I think this would be optional
 +
 +[23:39:52] <​jtruelove_>​ depending on how long it would take data before purging
 +
 +[23:40:11] <​temporalfox>​ but you can discuss that on vertx-dev if you are concerned
 +
 +[23:40:19] <​jtruelove_>​ or popping longest in the queue data etc..
 +
 +[23:44:27] <​jtruelove_>​ i'll try and engage on that thread