IT数码 购物 网址 头条 软件 日历 阅读 图书馆
TxT小说阅读器
↓语音阅读,小说下载,古典文学↓
图片批量下载器
↓批量下载图片,美女图库↓
图片自动播放器
↓图片自动播放器↓
一键清除垃圾
↓轻轻一点,清除系统垃圾↓
开发: C++知识库 Java知识库 JavaScript Python PHP知识库 人工智能 区块链 大数据 移动开发 嵌入式 开发工具 数据结构与算法 开发测试 游戏开发 网络协议 系统运维
教程: HTML教程 CSS教程 JavaScript教程 Go语言教程 JQuery教程 VUE教程 VUE3教程 Bootstrap教程 SQL数据库教程 C语言教程 C++教程 Java教程 Python教程 Python3教程 C#教程
数码: 电脑 笔记本 显卡 显示器 固态硬盘 硬盘 耳机 手机 iphone vivo oppo 小米 华为 单反 装机 图拉丁
 
   -> 大数据 -> Ignite Compute helloworld-分布式计算 -> 正文阅读

[大数据]Ignite Compute helloworld-分布式计算

目录

1. 背景

2. 步骤

2.1. 在pom.xml里加ignite依赖

2.2.? Ignite compute server 1

2.2.? Ignite compute server 2

2.3.? Ignite compute client


1. 背景

利用Ignite做一个分布式计算。1个Ignite Client 和 2 ignite server.

2. 步骤

2.1. 在pom.xml里加ignite依赖

<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
 	<ignite.version>2.11.1</ignite.version>
  </properties>
  <dependencies>
  		 <dependency>
            <groupId>org.apache.ignite</groupId>
            <artifactId>ignite-core</artifactId>
            <version>${ignite.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.ignite</groupId>
            <artifactId>ignite-spring</artifactId>
            <version>${ignite.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.ignite</groupId>
            <artifactId>ignite-indexing</artifactId>
            <version>${ignite.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.ignite</groupId>
            <artifactId>ignite-web</artifactId>
            <version>${ignite.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.ignite</groupId>
            <artifactId>ignite-ssh</artifactId>
            <version>${ignite.version}</version>
        </dependency>
  </dependencies>

2.2.? Ignite compute server 1

import java.util.Collections;
import org.apache.ignite.Ignite;
import org.apache.ignite.Ignition;
import org.apache.ignite.configuration.IgniteConfiguration;
import org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi;
import org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder;
 
public class IgniteComputeServer1 {
 
    public static void main(String[] args){
        // Preparing IgniteConfiguration using Java APIs
        IgniteConfiguration cfg = new IgniteConfiguration();

        // Classes of custom Java logic will be transferred over the wire from this app.
        cfg.setPeerClassLoadingEnabled(true);

        // Setting up an IP Finder to ensure the client can locate the servers.
        TcpDiscoveryMulticastIpFinder ipFinder = new TcpDiscoveryMulticastIpFinder();
        ipFinder.setAddresses(Collections.singletonList("127.0.0.1:47500..47509"));
        cfg.setDiscoverySpi(new TcpDiscoverySpi().setIpFinder(ipFinder));

        System.out.println("starting ignition...");
        Ignite ignite = Ignition.start(cfg);
    	System.out.println("ignite server started");
    }
}

运行结果

[22:45:22] To start Console Management & Monitoring run ignitevisorcmd.{sh|bat}
[22:45:22] 
[22:45:22] Ignite node started OK (id=866b923f)
[22:45:22] Topology snapshot [ver=1, locNode=866b923f, servers=1, clients=0, state=ACTIVE, CPUs=16, offheap=3.2GB, heap=3.5GB]
[22:45:22]   ^-- Baseline [id=0, size=1, online=1, offline=0]
ignite server started

2.2.? Ignite compute server 2

import java.util.Collections;
import org.apache.ignite.Ignite;
import org.apache.ignite.Ignition;
import org.apache.ignite.configuration.IgniteConfiguration;
import org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi;
import org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder;
 
public class IgniteComputeServer2 {
 
    public static void main(String[] args){
        IgniteConfiguration cfg = new IgniteConfiguration();
        cfg.setPeerClassLoadingEnabled(true);
        TcpDiscoveryMulticastIpFinder ipFinder = new TcpDiscoveryMulticastIpFinder();
        ipFinder.setAddresses(Collections.singletonList("127.0.0.1:47500..47509"));
        cfg.setDiscoverySpi(new TcpDiscoverySpi().setIpFinder(ipFinder));
        System.out.println("starting ignition...");
        Ignite ignite = Ignition.start(cfg);
    	System.out.println("ignite server started");
    }
}

运行结果

[22:45:22] Ignite node started OK (id=866b923f)
[22:45:22] Topology snapshot [ver=1, locNode=866b923f, servers=1, clients=0, state=ACTIVE, CPUs=16, offheap=3.2GB, heap=3.5GB]
[22:45:22]   ^-- Baseline [id=0, size=1, online=1, offline=0]
ignite server started

2.3.? Ignite compute client

import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections;
import org.apache.ignite.Ignite;
import org.apache.ignite.Ignition;
import org.apache.ignite.configuration.IgniteConfiguration;
import org.apache.ignite.lang.IgniteCallable;
import org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi;
import org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder;
 
public class IgniteClient {
    public static void main(String[] args){
    	Ignition.setClientMode(true);
        IgniteConfiguration cfg = new IgniteConfiguration();
        cfg.setPeerClassLoadingEnabled(true);
        TcpDiscoveryMulticastIpFinder ipFinder = new TcpDiscoveryMulticastIpFinder();
        ipFinder.setAddresses(Collections.singletonList("127.0.0.1:47500..47509"));
        cfg.setDiscoverySpi(new TcpDiscoverySpi().setIpFinder(ipFinder));     
        Ignite ignite = Ignition.start(cfg);
        Collection<IgniteCallable<Integer>> calls = new ArrayList<>();
	    // Iterate through all the words in the sentence and create Callable jobs.
	    for (final String word : "Count characters using callable".split(" ")) {
	        calls.add(new IgniteCallable<Integer>() {
	            @Override public Integer call() throws Exception {
	            	System.out.println("IgniteCallable:"+word);
	                return word.length();
	            }
	        });
	    }
	    // Execute collection of Callables on the grid.
	    Collection<Integer> res = ignite.compute().call(calls);
	    int sum = 0;
	    // Add up individual word lengths received from remote nodes.
	    for (int len : res)
	        sum += len;
	    System.out.println(">>> Total number of characters in the phrase is '" + sum + "'.");    
    }
}

运行结果

Ignite client 把4个线程分发到两个Ignite compute server 上。计算完后返回回来

[22:46:35] Ignite node started OK (id=65c8572d)
[22:46:35] Topology snapshot [ver=3, locNode=65c8572d, servers=2, clients=1, state=ACTIVE, CPUs=16, offheap=6.4GB, heap=11.0GB]
[22:46:35]   ^-- Baseline [id=0, size=2, online=2, offline=0]
>>> Total number of characters in the phrase is '28'.

Ignite compute server 1执行了两个线程

[22:46:20] Joining node doesn't have stored group keys [node=d99385c3-6303-4366-bbd2-df4847d6bbb2]
[22:46:20] Topology snapshot [ver=2, locNode=866b923f, servers=2, clients=0, state=ACTIVE, CPUs=16, offheap=6.4GB, heap=7.1GB]
[22:46:20]   ^-- Baseline [id=0, size=2, online=2, offline=0]
[22:46:35] Topology snapshot [ver=3, locNode=866b923f, servers=2, clients=1, state=ACTIVE, CPUs=16, offheap=6.4GB, heap=11.0GB]
[22:46:35]   ^-- Baseline [id=0, size=2, online=2, offline=0]
IgniteCallable:Count
IgniteCallable:using

Ignite compute server 2执行了两个线程

ignite server started
[22:46:35] Topology snapshot [ver=3, locNode=d99385c3, servers=2, clients=1, state=ACTIVE, CPUs=16, offheap=6.4GB, heap=11.0GB]
[22:46:35]   ^-- Baseline [id=0, size=2, online=2, offline=0]
IgniteCallable:characters
IgniteCallable:callable

  大数据 最新文章
实现Kafka至少消费一次
亚马逊云科技:还在苦于ETL?Zero ETL的时代
初探MapReduce
【SpringBoot框架篇】32.基于注解+redis实现
Elasticsearch:如何减少 Elasticsearch 集
Go redis操作
Redis面试题
专题五 Redis高并发场景
基于GBase8s和Calcite的多数据源查询
Redis——底层数据结构原理
上一篇文章      下一篇文章      查看所有文章
加:2022-03-03 16:21:05  更:2022-03-03 16:25:08 
 
开发: C++知识库 Java知识库 JavaScript Python PHP知识库 人工智能 区块链 大数据 移动开发 嵌入式 开发工具 数据结构与算法 开发测试 游戏开发 网络协议 系统运维
教程: HTML教程 CSS教程 JavaScript教程 Go语言教程 JQuery教程 VUE教程 VUE3教程 Bootstrap教程 SQL数据库教程 C语言教程 C++教程 Java教程 Python教程 Python3教程 C#教程
数码: 电脑 笔记本 显卡 显示器 固态硬盘 硬盘 耳机 手机 iphone vivo oppo 小米 华为 单反 装机 图拉丁

360图书馆 购物 三丰科技 阅读网 日历 万年历 2024年11日历 -2024/11/24 10:58:09-

图片自动播放器
↓图片自动播放器↓
TxT小说阅读器
↓语音阅读,小说下载,古典文学↓
一键清除垃圾
↓轻轻一点,清除系统垃圾↓
图片批量下载器
↓批量下载图片,美女图库↓
  网站联系: qq:121756557 email:121756557@qq.com  IT数码