先更新系统和安装 pip3
sudo apt update
sudo apt upgrade
sudo apt install python-setuptools
sudo apt install python3-pip安装基本库
sudo apt install libmysqlclient-dev
sudo apt install libssl-dev
sudo apt install libkrb5-dev更新本地 pip3 package
注: 要加 ~/.profile export PATH=/home/kazuya/.Local/bin:$PATH
pip3 list --outdated
pip3 list --outdated --format=freeze | grep -v '^\-e' | cut -d = -f 1 | xargs -n1 pip3 install -U安装 python 虚拟环境
sudo apt install python3-virtualenv建立 airflow 虚拟环境 和 数据存放 目录
sudo mkdir /opt/airflow
sudo chown -R kazuya:kazuya /opt/airflow输出 AIRFLOW_HOME
vi ~/.profile
export AIRFLOW_HOME=/opt/airflow建立 airflow 虚拟环境 ?
sudo virtualenv /opt/airflow_env
sudo chown -R kazuya:kazuya /opt/airflow_env起动 python 虚拟环境
cd /opt/airflow_env/bin
source activate更新虚拟环境 pip3 package 和 安装 apache-airflow
pip3 list --outdated
pip3 list --outdated --format=freeze | grep -v '^\-e' | cut -d = -f 1 | xargs -n1 pip3 install -U
pip3 install apache-airflow
pip3 install typing_extensions查看 apache-airflow?安装路径 Location: /opt/airflow_env/lib/python3.8/site-packages
$ pip3 show apache-airflow
Name: apache-airflow
Version: 2.1.4
Summary: Programmatically author, schedule and monitor data pipelines
Home-page: https://airflow.apache.org/
Author: Apache Software Foundation
Author-email: dev@airflow.apache.org
License: Apache License 2.0
Location: /opt/airflow_env/lib/python3.8/site-packages
Requires: flask-wtf, dill, python-dateutil, sqlalchemy, sqlalchemy-jsonfield, pendulum, python-nvd3, markdown, apache-airflow-providers-ftp, pygments, docutils, cattrs, pyjwt, importlib-metadata, termcolor, pandas, argcomplete, apache-airflow-providers-imap, numpy, blinker, openapi-spec-validator, setproctitle, httpx, markupsafe, importlib-resources, flask-caching, tabulate, psutil, gunicorn, iso8601, alembic, pyyaml, attrs, croniter, flask-appbuilder, jsonschema, python-daemon, flask, swagger-ui-bundle, lazy-object-proxy, itsdangerous, jinja2, apache-airflow-providers-http, python3-openid, flask-login, cryptography, colorlog, marshmallow-oneofschema, unicodecsv, clickclick, python-slugify, tenacity, inflection, lockfile, rich, apache-airflow-providers-sqlite, werkzeug, graphviz
Required-by:初始数据库和建立管理账号
airflow db init
airflow users create --username admin --firstname kazuya --lastname kazuya --role Admin --email XXXXXXX@gmail.com
Password起动 airflow 排程器
airflow scheduler再开一个 session 和 source activate 虚拟环境 起动 airflow 服务器
airflow webserver -p 8080
?
?
Ctrl-C 停止服务
Ctrl-C to quit webserver
Ctrl-C to quit scheduler退出 python 虚拟环境
deactivate