Publish to my blog (weekly)

• pattern matching - Scala: How to use case keyword without match keyword? - Stack Overflow

  • It's just syntactic sugar. In scala, you can use any method that takes a single parameter as a binary operator.

• scala - How to serialize/deserialize case classes to/from Json in Play 2.1 - Stack Overflow

  • Json combinators doesn't work for single field case class in Play 2.1 (it should be possible in 2.2)
  • case class MyType(value: String) val myTypeRead = (__ \ 'value).read[String].map(v => MyType(v)) // covariant map

• ScalaAkka - 2.5.x

  • The actorOf method is used to create a new actor. Notice that we’ve declared this controller to be a singleton. This is necessary since we are creating the actor and storing a reference to it, if the controller was not scoped as singleton, this would mean a new actor would be created every time the controller was created, which would ultimate throw an exception because you can’t have two actors in the same system with the same name.

• Dynamic Akka Streams Using Stage Actors - Lucid Electric Dreams

  • Getting around this limitation
    • 이 한계를 극복하기
  • Akka Streams are materialised and run on top of actors but the DSL (understandably) completely hides this fact from the implementers. Letting users directly interact with actors could lead to inconsistent behaviour and even destabilise the entire pipeline. However, in the recent releases of Akka streams, users are now able to access a so-called stage actor from within their custom graph stages. Stage actors are not fully-fledged actors and only support a subset of operations on them .

• Integration • Akka Documentation

  • You should consider using Source.queue if you want a backpressured actor interface.

• Integration — Akka Documentation

  • • if (buf.isEmpty && totalDemand > 0)
    • onNext(job)
  • case OnNext(Msg(id, replyTo)) =>

• Integrating Akka Streams and Akka Actors: Part II

  • sophisticated
    • 매우 복잡한adjective 매우 복잡한, 소박한 데가 없는, 섞음질한
  • compliment
    • 경의noun 경의, 경의의 표시, 의례적인 인사말verb 경의를 표하다, ...에게 물건을 중정하다, 보완하다.
  • An extremely important aspect to understand is that the materialized stream is running as a set of actors on the threads of the execution context on which they were allocated. In other words, the stream is running independently from the actor that allocated it. This becomes very important if the stream is long-running, or even infinite, and we want the actor to manage the life-cycle of the stream, such that when the actor stops, the stream is terminated.
  • (killSwitch, done)
  • .viaMat(KillSwitches.single)(Keep.right)
  • done.map(_ => self ! "done")
  • killSwitch.shutdown()
  • The WebSocket client is implemented as an Akka Stream, therefore, the WindTurbineSimulator actor is an actor encapsulating a stream.
  • .throttle(  elements = 100,  per = 1 second,  maximumBurst = 100,  mode = ThrottleMode.shaping  )
  • 1) complete the stream with a failure;
  • 2) drop the element and continue processing;
  • 3) drop the element and continue the stream after restarting the stage, discarding any intermediate state.
  • I demonstrated how Akka Actors compliment the Akka Stream API for fault tolerance and the life-cycle management of streams.

• Patterns for Streaming Measurement Data with Akka Streams

  • The following example simulates an infinite stream of samples and writes them to a database, efficiently, 1000 elements at at time.
  • .grouped(1000)
  • Grouping often introduces an unacceptable latency. To address this, the Akka Streams API has the groupedWithin mechanism to group events, but also emit events within a bounded time-frame, even if the maximum number of events has not been satisfied.
  • .groupedWithin(1000, 100 milliseconds)
  • constituents
    • 구성 성분noun 성분, 선거권자, 대리 지정인
  • .map(MessageParser.parse)
  • This works, but decomposing messages can usually be done in a single step, just using mapConcat to emit individual elements downstream.
  • .mapConcat(MessageParser.parse)
  • overwhelming
    • 압도적 인adjective 압도적인
  • when the streams are unbounded and the message rates are dynamic. Akka Streams provides a seamless way to do this, with the added benefit that if the number of outstanding requests is saturated, the stream will backpressure and not request additional events, rather than exhausting memory.
  • It limits the number of outstanding requests to the database from this client to a maximum of 10.
  • groupedWithin(1000, 100 milliseconds)
  • .mapAsync(10)(database.bulkInsertAsync)
  • If the downstream order of elements is not important, Akka Streams also provides the mapAsyncUnordered mechanism to limit the rate, without necessarily preserving downstream order.
  • .mapAsyncUnordered(10)(database.bulkInsertAsync)
  • defect
    • 결함noun 결함, 결손, 결점
  • If the upstream exceeds the specified rate, the throttle element can fail the stream, or shape the stream by backpressuring.
  • The following example limits the upstream to sending only one message per second, even though the upstream is attempting to emit elements faster than this.
  • .throttle(elements = 1, per = 1 second, maximumBurst = 1, mode = ThrottleMode.shaping)
  • The maximumBurst parameter can be used to allow the client to send a burst of messages, while still respecting the throttle once this upper bound has been reached.
  • Akka Streams executes streams sequentially, on a single thread.
  • .async
  • which operations are CPU-bound,
  • which ones are memory-bound,
  • which ones can be executed asynchronously,
  • but unlike traditionally multi-threaded programming, implementing asynchronous boundaries is as simple, and as thread-safe, as adding an async element
  • The first is reclaiming resources when a stream is idle.
  • decommissioned
    • 폐기 된
  • .idleTimeout(1 minute)
  • recover {  case ex: java.util.concurrent.TimeoutException =>  logger.error(s"Device $id has been idle for 1 minute")  }
  • The other situation in which I find enforcing idle timeouts enlightening is when developing functional tests for streaming-data systems.
  • The watchTermination element can be used to take an action when the upstream competes with success or failure
  • greeter.watchTermination() { (_, done) =>  done.onComplete {  case Success(_) =>  logger.info("Completed successfully")  Stats.stats.updateAverageSessionLength(System.currentTimeMillis() - startTime)  case Failure(ex) =>  logger.error(s"Completed with failure : $ex")  Stats.stats.updateAverageSessionLength(System.currentTimeMillis() - startTime)  }
  • handleWebSocketMessages(greeterRoute)
  • batching measurements
  • decomposing measurements
  • rate-limiting requests
  • throttling requests
  • performing tasks concurrently

• About Akka Streams – Zalando Tech Blog

  • Sinks (one input)
  • Sources (one output)
  • Flows (one input and one output)
  • a single string value goes from its Source through a mapping stage Flow[String].map and ends up in a Sink that printlns its input.
  • We have just created a RunnableGraph, which is kind of a blueprint, and any other (arbitrary complex) streams are only blueprints as well.
  • To execute, materialize (in Akka Streams’ terms) them, we need a Materializer — a special tool that actually runs streams, allocating all resources that are necessary and starting all the mechanics. It is theoretically possible to have any kind of Materializer, but out of the box, the library includes only one, ActorMaterializer. It executes stream stages on top of Akka actors.
  • Various Sources (like Source.fromIterator, Source.queue, Source.actorRef, etc.)
  • Various Sinks (like Sink.head, Sink.fold, Sink.actorRef, etc.)
  • Various Flows (like Flow.filter, Flow.fold. Flow.throttle, Flow.mapAsync, Flow.delay, Flow.merge, Flow.broadcast, etc.)
  • Run a stream five times — get five materialized values, completely independent.
  • toMat takes the additional parameter combine, a function that combines two materialized values: one from the previous stage and one from the current stages.
  • kill switches. This is an object used externally to stop the materialization of a stream.
  • Another stage we added is KillSwitches.single, which just creates a kill switch per materialization (not shared between materializations) as its materialized values

• ScalaWebSockets - 2.6.x

  • Play’s WebSocket handling mechanism is built around Akka streams. A WebSocket is modelled as a Flow, incoming WebSocket messages are fed into the flow, and messages produced by the flow are sent out to the client.
  • a flow is often viewed as something that receives messages, does some processing to them, and then produces the processed messages - there is no reason why this has to be the case, the input of the flow may be completely disconnected from the output of the flow. Akka streams provides a constructor, Flow.fromSinkAndSource, exactly for this purpose, and often when handling WebSockets, the input and output will not be connected at all.
  • Note that ActorFlow.actorRef(...) can be replaced with any Akka Streams Flow[In, Out, _], but actors are generally the most straightforward way to do it.
  • Any messages received from the client will be sent to the actor, and any messages sent to the actor supplied by Play will be sent to the client.
  • When the WebSocket has closed, Play will automatically stop the actor.
  • to clean up any resources the WebSocket might have consumed.
  • Play will automatically close the WebSocket when your actor that handles the WebSocket terminates
  • to close the WebSocket, send a PoisonPill to your own actor:
  • Play provides acceptOrResult to address this, allowing you to return either a result (such as forbidden, or not found), or the actor to handle the WebSocket with:
  • JsValue, JsValue
  • You may have noticed that there are two type parameters, this allows us to handle differently typed messages coming in to messages going out.
  • Actors are not always the right abstraction for handling WebSockets, particularly if the WebSocket behaves more like a stream.
  • val in = Sink.foreach[String](println)
  • val out = Source.single("Hello!").concat(Source.maybe)
  • Flow.fromSinkAndSource(in, out)
  • discards the input data and closes the socket just after sending the Hello! message:
    • 브라우저 --> 서버 (X)브라우저 <-- 서버 (O)

• JavaWebSockets - 2.6.x

  • Play’s WebSocket handling mechanism is built around Akka streams
  • A WebSocket is modelled as a Flow, incoming WebSocket messages are fed into the flow, and messages produced by the flow are sent out to the client.
  • a flow is often viewed as something that receives messages, does some processing to them, and then produces the processed messages - there is no reason why this has to be the case, the input of the flow may be completely disconnected from the output of the flow
  • Akka streams provides a constructor, Flow.fromSinkAndSource, exactly for this purpose, and often when handling WebSockets, the input and output will not be connected at all.

• Akka Streams: A Motivating Example

  • For the purposes of demonstration, consider a prototype application that serves WebSocket connections. Clients use the WebSocket connection to stream measurement data to the server. The server parses the incoming messages and writes the measurements to a database. The rate at which messages are sent to the server is extremely variable. There are bursts where the message rate is extremely high, but there are also periods where very few messages are sent to the server. The server is not in control of the rate at which a client sends messages to the server.
  • message-based
  • concurrent
  • scalable
  • fault-tolerant
  • A drawback of this approach, however, is that since new messages arrive at a variable rate, there are some periods of time where messages are buffered and the samples in the database are not current for minutes or more. This can be addressed by scheduling a database insert at least once-a-second, even if 1000 messages have not been received.
  • This approach is somewhat inefficient in that the Database Actor will write messages to the database every second, as long as there are queued messages, even if a write to the database has just occurred.
  • overwhelms
    • 압도하다, 물속에 가라앉히다, 압승하다
  • the implementation of the Database Actor is now very complex. I would not be surprised if this implementation has some subtle bugs. Testing all of the edge cases will be difficult. To complicate matters further, I haven't even considered error handling. Error handling will add significant complexity, especially if the ordering of the messages is important and must be maintained.
  • throttles
    • 조절 장치명사: 조절판, 목구멍동사: 누르다, 목을 조르다, 절기하다
  • Implementing some form of flow control with the client is a difficult task and it will require a significant amount of work and testing. In addition, the problems I'm addressing here—batching requests, limiting concurrency, scheduling events, flow control—are common problems when dealing with streaming data, especially unbounded streams, and are not specific to this application. These are the motivations behind the Akka Streams API.
  • rate-limiting requests
  • batching messages
  • handling flow control
  • will not buffer messages for more than one second
  • will limit the number of
  • outstanding
    • 두드러진형용사: 두드러진, 미불인, 부채 따위가 미불인
  • asynchronous writes to the database to 10
  • resemblance
    • 유사, 외관, 닮은 얼굴
  • steep
    • 험한, 터무니없는

• Integrating Akka Streams and Akka Actors: Part I

  • fault tolerance
  • concurrency
  • distributed computation
  • out-of-memory exceptions
  • poor performance
  • flow control
  • discouraging
    • 낙담한, 낙담시키는
  • complimentary
    • 무료adjective 칭찬하는, 무료의
  • he WebSocket server is designed to handle connections from a large number of clients. Each client emits an unbounded stream of measurements. The measurementsWebSocket stream computes intermediate sums and sends these intermediate results to the Total actor, at least every second.
  • This is a simple example of how the Akka Streams API can be used to handle the processing of unbounded streaming-data, while an actor can be used to maintain mutable state.
  • subtle
    • 세밀한adjective 세밀한, 교활한, 묽은
  • In other words, there is a discontinuity between the flow-controlled, unbounded stream-processing offered by the Akka Streams API, and the asynchronous messaging of actors, which is not flow-controlled. In addition to performance problems, this can lead to out-of-memory exceptions, as I detailed in my essay motivating the need for the Akka Streams API.
  • Probably the most straightforward and flexible way to achieve flow control between streams and actors is to use the ask pattern, to asynchronously send a message to an actor, combined with the mapAsync stage to await the response, before sending additional messages to the actor.
  • mapAsync(1) {
  • (total ? Increment(measurements.sum))  .mapTo[Done]  .map(_ => lastMessage)
  • sender ! Done
  • If the actor you are interacting with is essentially the termination point of the stream, it can be treated as a fully backpressured sink, using Sink.actorRefWithAck
  • As usual, the actor processes only one message at a time, but, in this case, the actor must respond with an acknowledgment message to support backpressure.
  • the actor must also acknowledge an initialization message, to indicate that it is ready to handle messages.
  • It may optionally handle a message when the stream is completed.
  • .alsoTo(sink)
  • val sink = Sink.actorRefWithAck(total, Init, Ack, Complete(id))  handleWebSocketMessages(measurementsWebSocket(sink))
  • The first is to use Source.actorRef. Messages sent to the actor materialized from this source will be emitted downstream, when there is demand.
  • hey will be buffered, up to the specified maximum, in conjunction with the overflow strategy.
  • The second approach is to use Source.queue, which is an improvement, since it can provide backpressure.
  • Source.queue
  • .mapAsync(1)(x => queue.offer(x))
  • I provided some introductory examples for integrating Akka Streams and Akka Actors.

• Crawling the Web with Akka Streams – Universal Language

  • Source is where the input data for the stream comes from. Source has output, but does not have input.
  • an actor
  • simply a collection
  • messaging queue listener
  • Sink is an endpoint which processes the results produced by the stream
  • send it somewhere else for further processing.
  • store the produced data
  • OverflowStrategy.backpressure means that our stream will backpressure the source until we have less than 100 elements left in the queue for processing.
  • Here, we define our own request strategy, that will accept elements from stream in batches of 10 (or less, if the producer requested a smaller number, which is represented by remainingRequested).

• pkinsky/akka-streams-example

  • An instance of the type Source[Out] produces a potentially unbounded stream of elements of type Out
  • mapConcat 'Transforms each input element into a sequence of output elements that is then flattened into the output stream'
  • Since we already have a Source[Seq[T]], we just pass the identity function to mapConcat.
  • ForeachSinks
  • A Sink[In] consumes elements of type In
  • FoldSinks
  • which fold some number of elements of type A into an initial value of type B using a function (A, B) => B, produces a Future[B] that completes when the stream completes.

• "크롬, 파이어폭스, 엣지에 질렸을 때" 갈아탈 신선한 대안 브라우저 10종 - ITWorld Korea

Posted from Diigo. The rest of my favorite links are here.