๋ฐ์ดํ„ฐ๋ถ„์„ 4

์‹ค์‹œ๊ฐ„ ๋ฐ์ดํ„ฐ ๋ถ„์„ ํ™˜๊ฒฝ ๊ตฌ์ถ• - 10. Streamlit, Clickhouse๋กœ ์‹ค์‹œ๊ฐ„ ๋Œ€์‹œ๋ณด๋“œ ๊ตฌํ˜„ ๐Ÿ”จ(part. 1)

๋ฐ์ดํ„ฐ ๋ถ„์„ ํ™˜๊ฒฝ ๊ตฌ์ถ• - 10. Streamlit, Clickhouse๋กœ ์‹ค์‹œ๊ฐ„ ๋Œ€์‹œ๋ณด๋“œ ๊ตฌํ˜„ ๐Ÿ”จ(part. 1)์ด๋ฒˆ ๊ธ€์—์„œ๋Š” ์ง€๊ธˆ๊นŒ์ง€ ๊ตฌ์ถ•ํ–ˆ๋˜ Clickhouse ์ธํ”„๋ผ์™€ Streamlit์„ ์—ฐ๊ฒฐํ•˜์—ฌ ์‹ค์‹œ๊ฐ„ ๋Œ€์‹œ๋ณด๋“œ๋ฅผ ๊ตฌํ˜„ํ•ด๋ณด๊ฒ ์Šต๋‹ˆ๋‹ค.์ฒซ๋ฒˆ์งธ ํŒŒํŠธ๋กœ Streamlit ๋‚ด Clickhouse ์—ฐ๊ฒฐ ํŒจํ‚ค์ง€ ์„ค์น˜์™€ ๋ฐ์ดํ„ฐ ์กฐํšŒ ํ”„๋กœ๊ทธ๋žจ์„ ๋งŒ๋“ค์–ด๋ณด๊ฒ ์Šต๋‹ˆ๋‹ค.- Clickhouse ์‹ค์‹œ๊ฐ„ ๋ถ„์„ ํ…Œ์ด๋ธ” ์ƒ์„ฑ(https://jongwho.tistory.com/26) ๋ฐ์ดํ„ฐ ๋ถ„์„ ํ™˜๊ฒฝ ๊ตฌ์ถ• - 05. Clickhouse ์‹ค์‹œ๊ฐ„ ๋ฐ์ดํ„ฐ ๋ถ„์„ ํ…Œ์ด๋ธ” ์ƒ์„ฑํ•˜๊ธฐ๐Ÿš€ ์ด ๊ธ€์—์„œ ๋‹ค๋ฃฐ ๋‚ด์šฉ1๏ธโƒฃ Kafka์™€ ์—ฐ๊ฒฐ๋œ ClickHouse ํ…Œ์ด๋ธ” ์ƒ์„ฑ (ํ™˜๊ฒฝ์„ค์ • ํฌํ•จ)2๏ธโƒฃ ์‹ค์Šต์šฉ ๋ฐ์ดํ„ฐ์…‹ kafka produce3๏ธโƒฃ Kafka ..

์‹ค์‹œ๊ฐ„ ๋ฐ์ดํ„ฐ ๋ถ„์„ ํ™˜๊ฒฝ ๊ตฌ์ถ• - 09. Streamlit ์†Œ๊ฐœ ๋ฐ ์„ค์น˜, ์‹ค์ œ ํ˜„์—… ์‚ฌ์šฉ๊ธฐ

์ด์ „์— ์†Œ๊ฐœํ–ˆ๋˜ Clickhouse์—์„œ ์‹ค์‹œ๊ฐ„ ๋ฐ์ดํ„ฐ ๋ถ„์„์„ ์œ„ํ•ด๋Œ€์‹œ๋ณด๋“œ๋ฅผ ๊ฐ€์žฅ ๋น ๋ฅด๊ฒŒ ์ƒ์„ฑํ•  ์ˆ˜ ์žˆ๋Š” Streamlit ์„œ๋น„์Šค์— ๋Œ€ํ•ด ์†Œ๊ฐœํ•ฉ๋‹ˆ๋‹ค.๊ทธ๋ฆฌ๊ณ  ์‹ค์ œ ํ˜„์—…์—์„œ Streamlit์œผ๋กœ ์„œ๋น„์Šค๋ฅผ ๊ตฌ์ถ•ํ–ˆ๋˜ ํ›„๊ธฐ๋„ ์งง๊ฒŒ ๊ณต์œ ํ•ฉ๋‹ˆ๋‹ค.   ๋ฐ์ดํ„ฐ ๋ถ„์„ ํ™˜๊ฒฝ ๊ตฌ์ถ• - 05. Clickhouse ์‹ค์‹œ๊ฐ„ ๋ฐ์ดํ„ฐ ๋ถ„์„ ํ…Œ์ด๋ธ” ์ƒ์„ฑํ•˜๊ธฐ๐Ÿš€ ์ด ๊ธ€์—์„œ ๋‹ค๋ฃฐ ๋‚ด์šฉ1๏ธโƒฃ Kafka์™€ ์—ฐ๊ฒฐ๋œ ClickHouse ํ…Œ์ด๋ธ” ์ƒ์„ฑ (ํ™˜๊ฒฝ์„ค์ • ํฌํ•จ)2๏ธโƒฃ ์‹ค์Šต์šฉ ๋ฐ์ดํ„ฐ์…‹ kafka produce3๏ธโƒฃ Kafka ํ…Œ์ด๋ธ”์—์„œ ๋ฐ์ดํ„ฐ ์…‹ ํ™•์ธ ๋ฐฉ๋ฒ•1. Kafka์™€ ์—ฐ๊ฒฐ๋œ ClickHouse ํ…Œ์ด๋ธ” ์ƒjongwho.tistory.com  ๐Ÿ› ๏ธ Streamlit ์„ค์น˜ ๋ฐ ์ฐจํŠธ ์ƒ์„ฑ ์•„๋ž˜ ๋ช…๋ น์–ด๋ฉด streamlit ์„ค์น˜๊ฐ€ ์™„๋ฃŒ๋ฉ๋‹ˆ๋‹ค.pip ..

์‹ค์‹œ๊ฐ„ ๋ฐ์ดํ„ฐ ๋ถ„์„ ํ™˜๊ฒฝ ๊ตฌ์ถ• - 03. Kafka ์„ค์น˜ ๋ฐ ์‹ค์‹œ๊ฐ„ ๋ฐ์ดํ„ฐ ์ŠคํŠธ๋ฆฌ๋ฐ ๊ตฌ์„ฑ

Kafka๋ฅผ ๊ตฌ์„ฑํ•˜์—ฌ ์‹ค์‹œ๊ฐ„ ์ŠคํŠธ๋ฆฌ๋ฐ ๋ฐ์ดํ„ฐ๋ฅผ Clickhouse๋กœ ์ฒ˜๋ฆฌํ•˜๋Š” ๋ถ„์„ ํ™˜๊ฒฝ์„ ๊ตฌ์ถ•ํ•ฉ๋‹ˆ๋‹ค. ๐Ÿ’ก ์ด ๊ธ€์—์„œ ๋‹ค๋ฃฐ ๋‚ด์šฉ:Kubernetes ํ™˜๊ฒฝ์—์„œ Kafka ์„ค์น˜ ๋ฐ ์„ค์ •Helm ์ฐจํŠธ๋ฅผ ํ™œ์šฉํ•œ Kafka ๋ฐฐํฌKafka ํ† ํ”ฝ ์ƒ์„ฑ ๋ฐ ๋ฐ์ดํ„ฐ ์ŠคํŠธ๋ฆฌ๋ฐ ํ…Œ์ŠคํŠธ1. Kafka๋ž€?Kafka๋Š” ๋Œ€์šฉ๋Ÿ‰ ์‹ค์‹œ๊ฐ„ ๋ฐ์ดํ„ฐ ์ŠคํŠธ๋ฆฌ๋ฐ์„ ์œ„ํ•œ ๋ถ„์‚ฐ ๋ฉ”์‹œ์ง• ํ”Œ๋žซํผ์ž…๋‹ˆ๋‹ค.ClickHouse์™€ ํ•จ๊ป˜ ์‚ฌ์šฉํ•˜๋ฉด ์‹ค์‹œ๊ฐ„ ๋ฐ์ดํ„ฐ ์ˆ˜์ง‘ ๋ฐ ๋ถ„์„์ด ๊ฐ€๋Šฅํ•ฉ๋‹ˆ๋‹ค.๐Ÿ“Œ Kafka ์ฃผ์š” ๊ฐœ๋…๊ฐœ๋… ์„ค๋ช…Producer๋ฐ์ดํ„ฐ๋ฅผ Kafka์— ๋ณด๋‚ด๋Š” ์—ญํ• ConsumerKafka์—์„œ ๋ฐ์ดํ„ฐ๋ฅผ ์ฝ๋Š” ์—ญํ• Topic๋ฐ์ดํ„ฐ๊ฐ€ ์ €์žฅ๋˜๋Š” ๋…ผ๋ฆฌ์  ๊ณต๊ฐ„BrokerKafka ์„œ๋ฒ„ (๋ฐ์ดํ„ฐ๋ฅผ ์ €์žฅํ•˜๊ณ  ๊ด€๋ฆฌ)*ZookeeperKafka์˜ ํด๋Ÿฌ์Šคํ„ฐ ์ƒํƒœ ๊ด€๋ฆฌKafka 4..

์‹ค์‹œ๊ฐ„ ๋ฐ์ดํ„ฐ ๋ถ„์„ ํ™˜๊ฒฝ ๊ตฌ์ถ• - 02. ์ฟ ๋ฒ„๋„คํ‹ฐ์Šค์— Clickhouse ์„ค์น˜ ๋ฐ ํด๋Ÿฌ์Šคํ„ฐ ๊ตฌ์„ฑ

์ด๋ฒˆ ๊ธ€์—์„œ๋Š” Kubernetes ํ™˜๊ฒฝ์—์„œ ClickHouse๋ฅผ ๋ฐฐํฌ ๋ฐ ์‹คํ–‰ํ•˜๋Š” ๋ฐฉ๋ฒ•์„ ์„ค๋ช…ํ•ฉ๋‹ˆ๋‹ค.Docker Desktop์˜ ๋‚ด์žฅ Kubernetes๋ฅผ ํ™œ์šฉํ•˜๊ณ , Helm ๋ฐ Bitnami Helm ์ฐจํŠธ๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ๊ฐ„ํŽธํ•˜๊ฒŒ ClickHouse๋ฅผ ๋ฐฐํฌํ•ฉ๋‹ˆ๋‹ค.1. ์‚ฌ์ „ ์ค€๋น„1) kubectl ์„ค์น˜ (Kubernetes CLI)kubectl์€ Kubernetes ํด๋Ÿฌ์Šคํ„ฐ๋ฅผ ๊ด€๋ฆฌํ•˜๋Š” ๋ช…๋ น์ค„ ๋„๊ตฌ์ž…๋‹ˆ๋‹ค.๐Ÿ”น macOSbrew install kubectl๐Ÿ”น Windows (PowerShell)choco install kubernetes-cliโœ… ์„ค์น˜ ํ™•์ธ:kubectl version --client2) Docker Desktop์—์„œ Kubernetes ํ™œ์„ฑํ™”Docker Desktop์—๋Š” Kubernetes๊ฐ€ ..