๋ฐ์ดํ„ฐ๋ถ„์„ 3

๋ฐ์ดํ„ฐ ๋ถ„์„ ํ™˜๊ฒฝ ๊ตฌ์ถ• - 09. Streamlit ์†Œ๊ฐœ ๋ฐ ์„ค์น˜, ์‹ค์ œ ํ˜„์—… ์‚ฌ์šฉ๊ธฐ

์ด์ „์— ์†Œ๊ฐœํ–ˆ๋˜ Clickhouse์—์„œ ์‹ค์‹œ๊ฐ„ ๋ฐ์ดํ„ฐ ๋ถ„์„์„ ์œ„ํ•ด๋Œ€์‹œ๋ณด๋“œ๋ฅผ ๊ฐ€์žฅ ๋น ๋ฅด๊ฒŒ ์ƒ์„ฑํ•  ์ˆ˜ ์žˆ๋Š” Streamlit ์„œ๋น„์Šค์— ๋Œ€ํ•ด ์†Œ๊ฐœํ•ฉ๋‹ˆ๋‹ค.๊ทธ๋ฆฌ๊ณ  ์‹ค์ œ ํ˜„์—…์—์„œ Streamlit์œผ๋กœ ์„œ๋น„์Šค๋ฅผ ๊ตฌ์ถ•ํ–ˆ๋˜ ํ›„๊ธฐ๋„ ์งง๊ฒŒ ๊ณต์œ ํ•ฉ๋‹ˆ๋‹ค.   ๋ฐ์ดํ„ฐ ๋ถ„์„ ํ™˜๊ฒฝ ๊ตฌ์ถ• - 05. Clickhouse ์‹ค์‹œ๊ฐ„ ๋ฐ์ดํ„ฐ ๋ถ„์„ ํ…Œ์ด๋ธ” ์ƒ์„ฑํ•˜๊ธฐ๐Ÿš€ ์ด ๊ธ€์—์„œ ๋‹ค๋ฃฐ ๋‚ด์šฉ1๏ธโƒฃ Kafka์™€ ์—ฐ๊ฒฐ๋œ ClickHouse ํ…Œ์ด๋ธ” ์ƒ์„ฑ (ํ™˜๊ฒฝ์„ค์ • ํฌํ•จ)2๏ธโƒฃ ์‹ค์Šต์šฉ ๋ฐ์ดํ„ฐ์…‹ kafka produce3๏ธโƒฃ Kafka ํ…Œ์ด๋ธ”์—์„œ ๋ฐ์ดํ„ฐ ์…‹ ํ™•์ธ ๋ฐฉ๋ฒ•1. Kafka์™€ ์—ฐ๊ฒฐ๋œ ClickHouse ํ…Œ์ด๋ธ” ์ƒjongwho.tistory.com  ๐Ÿ› ๏ธ Streamlit ์„ค์น˜ ๋ฐ ์ฐจํŠธ ์ƒ์„ฑ ์•„๋ž˜ ๋ช…๋ น์–ด๋ฉด streamlit ์„ค์น˜๊ฐ€ ์™„๋ฃŒ๋ฉ๋‹ˆ๋‹ค.pip ..

๋ฐ์ดํ„ฐ ๋ถ„์„ ํ™˜๊ฒฝ ๊ตฌ์ถ• - 03. Kafka ์„ค์น˜ ๋ฐ ์‹ค์‹œ๊ฐ„ ๋ฐ์ดํ„ฐ ์ŠคํŠธ๋ฆฌ๋ฐ ๊ตฌ์„ฑ

Kafka๋ฅผ ๊ตฌ์„ฑํ•˜์—ฌ ์‹ค์‹œ๊ฐ„ ์ŠคํŠธ๋ฆฌ๋ฐ ๋ฐ์ดํ„ฐ๋ฅผ Clickhouse๋กœ ์ฒ˜๋ฆฌํ•˜๋Š” ๋ถ„์„ ํ™˜๊ฒฝ์„ ๊ตฌ์ถ•ํ•ฉ๋‹ˆ๋‹ค. ๐Ÿ’ก ์ด ๊ธ€์—์„œ ๋‹ค๋ฃฐ ๋‚ด์šฉ:Kubernetes ํ™˜๊ฒฝ์—์„œ Kafka ์„ค์น˜ ๋ฐ ์„ค์ •Helm ์ฐจํŠธ๋ฅผ ํ™œ์šฉํ•œ Kafka ๋ฐฐํฌKafka ํ† ํ”ฝ ์ƒ์„ฑ ๋ฐ ๋ฐ์ดํ„ฐ ์ŠคํŠธ๋ฆฌ๋ฐ ํ…Œ์ŠคํŠธ1. Kafka๋ž€?Kafka๋Š” ๋Œ€์šฉ๋Ÿ‰ ์‹ค์‹œ๊ฐ„ ๋ฐ์ดํ„ฐ ์ŠคํŠธ๋ฆฌ๋ฐ์„ ์œ„ํ•œ ๋ถ„์‚ฐ ๋ฉ”์‹œ์ง• ํ”Œ๋žซํผ์ž…๋‹ˆ๋‹ค.ClickHouse์™€ ํ•จ๊ป˜ ์‚ฌ์šฉํ•˜๋ฉด ์‹ค์‹œ๊ฐ„ ๋ฐ์ดํ„ฐ ์ˆ˜์ง‘ ๋ฐ ๋ถ„์„์ด ๊ฐ€๋Šฅํ•ฉ๋‹ˆ๋‹ค.๐Ÿ“Œ Kafka ์ฃผ์š” ๊ฐœ๋…๊ฐœ๋… ์„ค๋ช…Producer๋ฐ์ดํ„ฐ๋ฅผ Kafka์— ๋ณด๋‚ด๋Š” ์—ญํ• ConsumerKafka์—์„œ ๋ฐ์ดํ„ฐ๋ฅผ ์ฝ๋Š” ์—ญํ• Topic๋ฐ์ดํ„ฐ๊ฐ€ ์ €์žฅ๋˜๋Š” ๋…ผ๋ฆฌ์  ๊ณต๊ฐ„BrokerKafka ์„œ๋ฒ„ (๋ฐ์ดํ„ฐ๋ฅผ ์ €์žฅํ•˜๊ณ  ๊ด€๋ฆฌ)*ZookeeperKafka์˜ ํด๋Ÿฌ์Šคํ„ฐ ์ƒํƒœ ๊ด€๋ฆฌKafka 4..

๋ฐ์ดํ„ฐ ๋ถ„์„ ํ™˜๊ฒฝ ๊ตฌ์ถ• - 02. ์ฟ ๋ฒ„๋„คํ‹ฐ์Šค์— Clickhouse ์„ค์น˜ ๋ฐ ํด๋Ÿฌ์Šคํ„ฐ ๊ตฌ์„ฑ

์ด๋ฒˆ ๊ธ€์—์„œ๋Š” Kubernetes ํ™˜๊ฒฝ์—์„œ ClickHouse๋ฅผ ๋ฐฐํฌ ๋ฐ ์‹คํ–‰ํ•˜๋Š” ๋ฐฉ๋ฒ•์„ ์„ค๋ช…ํ•ฉ๋‹ˆ๋‹ค.Docker Desktop์˜ ๋‚ด์žฅ Kubernetes๋ฅผ ํ™œ์šฉํ•˜๊ณ , Helm ๋ฐ Bitnami Helm ์ฐจํŠธ๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ๊ฐ„ํŽธํ•˜๊ฒŒ ClickHouse๋ฅผ ๋ฐฐํฌํ•ฉ๋‹ˆ๋‹ค.1. ์‚ฌ์ „ ์ค€๋น„1) kubectl ์„ค์น˜ (Kubernetes CLI)kubectl์€ Kubernetes ํด๋Ÿฌ์Šคํ„ฐ๋ฅผ ๊ด€๋ฆฌํ•˜๋Š” ๋ช…๋ น์ค„ ๋„๊ตฌ์ž…๋‹ˆ๋‹ค.๐Ÿ”น macOSbrew install kubectl๐Ÿ”น Windows (PowerShell)choco install kubernetes-cliโœ… ์„ค์น˜ ํ™•์ธ:kubectl version --client2) Docker Desktop์—์„œ Kubernetes ํ™œ์„ฑํ™”Docker Desktop์—๋Š” Kubernetes๊ฐ€ ..