python pandas dataframe์œผ๋กœ bigquery insert ์‹œ ์˜ค๋ฅ˜ ๋ฐœ์ƒ

python์œผ๋กœ pandas๋กœ dataframe์„ ๋งŒ๋“ค์–ด bigquery์— insert๋ฅผ ํ•˜๋ฉด ์˜ค๋ฅ˜๊ฐ€ ๋ฐœ์ƒํ•˜๋Š” ๊ฒฝ์šฐ๊ฐ€ ์žˆ์Šต๋‹ˆ๋‹ค.

๋ฐ์ดํ„ฐ ํƒ€์ž…์ด ๋งž์ง€ ์•Š๋Š”๋‹ค๋Š” ์˜ค๋ฅ˜์ธ๋ฐ schema๋„ ํฌํ•จํ–ˆ๊ณ  autodetect = False ์˜ต์…˜๋„ ๋„ฃ์—ˆ์ง€๋งŒ ์†Œ์šฉ์ด ์—†์—ˆ์Šต๋‹ˆ๋‹ค.

๋Œ€๋žต ๊ฒ€์ƒ‰ํ•ด๋ณด๋‹ˆ autodetect=False ์˜ต์…˜์ด ๋จนํžˆ์ง€ ์•Š๊ณ  date๋‚˜ datetime ํƒ€์ž…์˜ ๋ฐ์ดํ„ฐ๋ฅผ STRING์ด ์•„๋‹Œ ๋‹ค๋ฅธ ํƒ€์ž…์œผ๋กœ ๋„ฃ์œผ๋ ค๊ณ  ํ• ๋•Œ ๋ฐœ์ƒํ•˜๋Š” ๋ฌธ์ œ ์ž…๋‹ˆ๋‹ค.

๋•Œ๋ฌธ์— schema๋ฅผ STRING์œผ๋กœ ํ•˜๋˜๊ฐ€ ์•„๋‹ˆ๋ฉด dataframe์„ ๋งŒ๋“ค๊ธฐ ์ „์— ๋ฐ์ดํ„ฐ๋ฅผ ๊ฐ์ฒด ํƒ€์ž…์œผ๋กœ ๋„ฃ์–ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.

ํŠน์ • json ํƒ€์ž…์˜ ํŒŒ์ผ์—์„œ ํŒŒ์‹ฑํ•˜๋ฉด์„œ dict ๋ฐ์ดํ„ฐ์— ๊ฐ์ฒด ํƒ€์ž…์œผ๋กœ ๋ณ€ํ™˜ํ•ด์„œ ๋„ฃ์Šต๋‹ˆ๋‹ค.

temp_data['createdAt'] = datetime.datetime.strptime(x['attributes']['createdAt'], '%Y-%m-%dT%H:%M:%SZ')

์ดํ›„ dataframe์œผ๋กœ ์ƒ์„ฑํ•ด์„œ bigqeury ์— ๋„ฃ์œผ๋ฉด ๋ฌธ์ œ ์—†์ด insert๊ฐ€ ๊ฐ€๋Šฅํ•ฉ๋‹ˆ๋‹ค.