Friday, 1 January 2016

KANBAN FLOW

First of all Wishing All a Very Happy and Prosperous 2016


Let's first gather details about the participants in KANBAN FLOW:

1) Processsing Node.
2) Kanban Tray.
3) Kanban Link.
4) Kanban Flow.
But the million dollar question is:

But What is Kanban Flow???

The Answer: A KanbanFlow is a network of dataflow connections made up from links known as Kanban Link

The Links hence can be arranged to take up the following forms:

1) chain (1:1)
2) fork (1:n)
3) funnel (n:1)
4) diamond (1:n:1)
5) acyclic directed graph.
However, Cycles are disallowed by default but can allowed by setting the cycleAllowed property to true.
Now if we take a closer look at the Kanban Link, then each link associates a Producer with a Consumer and each of these entities is being represented by Processsing Node

Now, each Processing Node consists of inputs, outputs, & a body and they are being represented as List, List, and a Closure respectively.

Now while creating the Kanban Link, the ouput of the producer becomes the input to consumer and vice versa and each Processing Node has an operator() which returns a Type of DataflowOperator, i.e. for a given Processing Node, after we invoke the operator(), then this DataflowProcessor waits for data to be present in all the elements if the Input List of the Processing Node, and once there are present, data from the Elements of the Input List is passed on to the Elements of the Output List which is the altogether the generic behaviour of the Data flow Operator. (as described in previous post).

So the Output List which is the Input to another Processing Node, gets the data and hence Data keeps on flowing from one Processing Node to Another. Now in all these arrangements, where does Kanban Tray fits?

Kanban Tray actually plays the role of the carrier of the product around a Kanban Link . Because when we invoke the to() method of the KanbanLink , then this method set the inputs of the outputs of the ProcessingNode with the instances of DataflowQueue.

So finally when we call the start() method of KanbalFlow, then the call is delegated to addTray() method, (through addOptimalNumberOfTrays() ) which adds the KanbanTary to the upstream.
Finally when we start the processing node with the operator(), the body parameter of the Processing Node gets these Kanban Tray(s) which are being passed as decribed above, when the all the inputs to a ProcessingNode becomes available (following the Dataflow operator Principal).

So a Kanban Flow may consists of multiple Kanban Link(s) in the forms as mentioned above and these Links co-ordinate with each other with the help of Dataflows.

So in case of Kanban Flow we can say that it is basically a conglomerate of different Dataflow or basically GPars entities working together and providing a framework kind of thing for the classical Producer-Consumer problem with Concurrent support. Please correct me if I am wrong in any aspect of the above definition

Let's traverse through the following Code snippet:


import static groovyx.gpars.dataflow.ProcessingNode.*
import groovyx.gpars.dataflow.KanbanFlow
import groovyx.gpars.dataflow.KanbanTray

class KanbanFLOW {
 
 static def upcount = 0 
 static def downcount = 100
  
 static def producer =  node{ down, up -> 
         ++upcount
         --downcount
         down <<  "The Data Down is $upcount"
         up << "The Data Up is $downcount"
         }
 
 static def consumer = node{ up -> println  "Up: ${up?.take()}"
  }
 
 static def consumer2 = node{ up2 -> println  "Up2: ${up2?.take()}" }
 
 static main(args)
 {
  //producer.maxForks = 4
  def flow1 = new KanbanFlow()
  def link1 = flow1.link(producer).to(consumer)
  def link2 = flow1.link(producer).to(consumer2)
  //link1.to(consumer2)
   
  flow1.start()
  sleep(500)
  //flow1.stop()
 }
}


This is in short about Kanban Flow, Please share your views and comments and wishing all a great successful year ahead.
View Subhankar Paul's profile on LinkedIn

SyncDataFlowQueue KNOWHOW

First of all Wishing All a Very Happy and Prosperous 2016


Now lets get into something NEW!!! and interesting....... Kanban Flow.

Now before diving into Kanban Flow, FRIENDS I would like to draw your attention towards a interesting perhaps important incident while Dealing with SyncDataflowQueue.

If we want to populate more than one SyncDataflowQueue from the same Thread, the Thread goes into a wait state. Now if you ask me "HOW", I do not have the answer.
Lets describe it with a code excerpt:


class PipelineTest {

 static def defaultPGroup = new DefaultPGroup(3)//new DefaultPGroup(new ResizeablePool(false, 3))

 static SyncDataflowQueue syncQueue = new SyncDataflowQueue()

 /**
  * Changing SyncDataflowQueue to DataFlowQueue changes the entire
  * behaviour But Why????
  * 
  */

 static SyncDataflowQueue outputQueue = new SyncDataflowQueue()
 //static DataflowQueue outputQueue = new DataflowQueue()

 static def result

 class SyncQueueEntry implements Callable
 {

  DataflowQueue q2
  SyncQueueEntry(DataflowQueue q2)
  {
   this.q2 = q2
  }

  public String call() throws Exception {
   // TODO Auto-generated method stub
   println "**** The Task Thread is ${Thread.currentThread()} ******"
   Dataflow.task{
    q2 << "okkkkkkies and Onnneeeessssss"
   }
   " !!!!!!!!!!! Inter Data !!!!!!!!!"
  }

 }




 /* static def t1 = Dataflow.task{
  println "**** The Task Thread is ${Thread.currentThread().getName()} ******"
  outputQueue << "Another Sync Queue"
  }*/

 static def upperCase = { msg -> msg.toUpperCase()
 }

 static def append1 = {msg ->

  "Subhankar IN Action $msg"

 }

 static def save = {text ->
  //Just pretending to be saving the text to disk, database or whatever
  'Saving ' + text
 }

 static main(args)
 {
  println "**** The Main Thread is ${Thread.currentThread().getName()} ******"
  Promise jk
  //def jk
  result = syncQueue.chainWith(defaultPGroup,upperCase).chainWith(defaultPGroup,append1).chainWith(defaultPGroup,save) /*into outputQueue*/
  def pipeLineTest = new PipelineTest()
  SyncQueueEntry syncQueue3 = new SyncQueueEntry(pipeLineTest,outputQueue)
  def group = new NonDaemonPGroup(3)

  println "Sleeping Thread.........."
  sleep(3000)
  println "Waking Thread......."

  
  syncQueue << "Groovy"
  syncQueue << "Grails"
  
  //sleep(6000)
  
  /**
   * Makes the main Thread goes into wait state 
   * as more than One SyncDataflowQueue are being
   * populated from same Thread i.e. Main Thread
   */
  
  //outputQueue << "Data Manually entered in OutputQueueee"
  
  /**
   * But when SyncDataflowQueue is being populated from another
   * Thread
   * 
   */
  
  Dataflow.task
  {
   outputQueue << "Data Manually entered in OutputQueueee"
  }
  
  println "The OUTPUTQUEUE Result is: ${outputQueue.val}"

  println "The syncQueue Result1 is: ${result.val}"
  println "The syncQueue Result2 is: ${result.val}"
  


  try{
   syncQueue << "cloud"

   println "The syncQueue Result3 is: ${result.val}"
   
   
   /**
    * As the SyncDataflowQueue in the Callable Task is Populated 
    * through a separate Thread, so the Promise can be
    * accessed it is not going to get stuck after shutting
    * down the Thread group
    * 
    * If we want to do before shutting down the Thread Group,
    * We need to populate SyncDataflowQueue in the callable Task 
    * in a Separate Thread
    * 
    */
   
   
   jk = group.task(syncQueue3)

  }
  catch(e)
  {
   e.printStackTrace()
  }

  /*jk.then{
   println "The CALLABLE result is $it"
  }*/
  
  println "The promise is: ${jk.get()}"
  syncQueue << "Heroku"

  println "The syncQueue Result4 is: ${result.val}"
  println "The OutputQueue Result is: ${outputQueue.val}"
  
  
  group.shutdown()
  
  //println "The promise is: ${jk.get()}"
  println "****End The Main Thread is ${Thread.currentThread().getName()} ******"


 }

}


Here we have used chainWith.
We can refer to chainWith as a kind of Handler.
So when data is being populated in a SyncDataflowQueue, then the registered Handler for the SyncDataflowQueue (chainWith) gets invoked and returns another SyncDataflowQueue (result) which we have accessed here to get the resultant value from the Handler.

Now from the same main Thread when we are trying to populate outputQueue i.e. another SyncDataflowQueue, the main thread will get stuck.

So we have used another thread with Dataflow.task{...., to populate outputQueue from another thread, the same thing happens when we initiate the callble task with the help of ThreadGroup.

Please refer to the comments as provided in code snippets block.

KANBAN FLOW
View Subhankar Paul's profile on LinkedIn