{%hackmd SyQbZi4Mr %}
Elixir - Flow
===
by [PastLeo](https://pastleo.me)
---
### 據說 Elixir 的 concurrent/parallel 很強
#### 應該說很容易實做
### 今天來講 parallel
---
### 來計算密碼的字典攻擊該用哪些字
* [1000 common English words](https://www.ef.com/english-resources/english-vocabulary/top-1000-words/)
* [Bruteforce Database - Password dictionaries](https://github.com/duyetdev/bruteforce-database)
* `38650-password-sktorrent.txt`
```mermaid
graph TD
subgraph passwds
p1(38650 items)
end
subgraph words
w1(1000 items)
end
```
---
![enum result](https://i.imgur.com/9hqbAqQ.png?1)
---
![enum cpu](https://i.imgur.com/Vz60gkd.png)
![nope](https://i.imgur.com/met6IyU.png)
---
![only one cpu is used](https://img-9gag-fun.9cache.com/photo/a6oqnPm_700bwp.webp)
---
## Flow
[github.com/elixir-lang/flow](https://github.com/elixir-lang/flow)
---
### 改成多執行緒版本,就這麼簡單
```diff=
File.stream!(word_file, read_ahead: 100_000)
- |> Enum.flat_map(&tidy/1)
- |> Enum.map(&cnt_word(&1, pw_file))
- |> Enum.group_by(&elem(&1, 1))
- |> Enum.map(fn {word_length, words} ->
+ |> Flow.from_enumerable()
+ |> Flow.partition()
+ |> Flow.flat_map(&tidy/1)
+ |> Flow.map(&cnt_word(&1, pw_file))
+ |> Flow.group_by(&elem(&1, 1))
+ |> Flow.map(fn {word_length, words} ->
Enum.sort_by(words, &elem(&1, 2), &>=/2)
|> Enum.take(5)
|> (&{word_length, &1}).()
end)
+ |> Enum.to_list()
|> Enum.sort_by(&elem(&1, 0), &>=/2)
|> show_result()
```
---
![flow cpu](https://i.imgur.com/fFXlfc9.png)
![that's it](https://i.imgur.com/3SUu8kJ.png)
---
<table>
<tr>
<td>
![flow result](https://i.imgur.com/HsR8yPv.png?1)
2 cores laptop
</td>
<td>
![flow result](https://i.imgur.com/HKuu1OU.png)
4 cores PC
</td>
</tr>
</table>
---
### 如何做到的?
1. Elixir 從 Erlang 拿來的 process & OTP
2. GenStage - back pressure
3. Flow 漂亮的包裝
---
### Elixir 從 Erlang 拿來的 process & OTP
```elixir=
Process.spawn(...) # very cheap
send(...) # very cheap
```
---
### GenStage - back pressure
```mermaid
graph TD
producer(Data source)
producer-consumer(Data process)
consumer(Output data)
producer--data stream-->producer-consumer
producer-consumer--data steam-->consumer
```
---
### GenStage - back pressure
```mermaid
graph TD
producer(Data source / Producer)
producer-consumer(Data process / Producer Consumer)
consumer(Output data / Consumer)
producer--data stream-->producer-consumer
producer-consumer--data steam-->consumer
producer-consumer-.subscribe.->producer
consumer-.subscribe.->producer-consumer
```
---
### GenStage - back pressure
```mermaid
graph TD
producer(Data source / Producer)
producer-consumer-1(Data process / Producer Consumer)
producer-consumer-2(Data process / Producer Consumer)
consumer(Output data / Consumer)
producer--data stream / subscribe---producer-consumer-1
producer--data stream / subscribe---producer-consumer-2
producer-consumer-1--data steam / subscribe---consumer
producer-consumer-2--data steam / subscribe---consumer
```
---
### [Flow](https://github.com/elixir-lang/flow)
#### Computational parallel flows on top of __***GenStage***__
```mermaid
graph TD
subgraph Flow
g1(GenStage)
g2(GenStage)
g3(GenStage)
gm(...)
end
```
---
### 謝謝~
#### 歡迎有興趣的人來深入了解~
#### [github.com/pastleo/passwd-wordcnt](https://github.com/pastleo/passwd-wordcnt)
#### [commit of using flow](https://github.com/pastleo/passwd-wordcnt/commit/bdcd5758433404243545de1224563b1592ea2b83?diff=unified)
{"metaMigratedAt":"2023-06-14T18:04:04.582Z","metaMigratedFrom":"YAML","title":"Elixir 閃電秀: Flow","breaks":"true","contributors":"[{\"id\":\"0eb274f7-a3a4-4c8e-b0d4-e5c08eaf9e72\",\"add\":6214,\"del\":2609}]"}