Skip to content

Commit

Permalink
sync to maven central repo.
Browse files Browse the repository at this point in the history
  • Loading branch information
@ZhangJiupeng
ZhangJiupeng committed Apr 10, 2017
1 parent 7c610cf commit 37a2fff
Show file tree
Hide file tree
Showing 3 changed files with 153 additions and 80 deletions.
4 changes: 2 additions & 2 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

Gospy is a flexible web crawler framework that allows you to develop complete web crawlers in few minutes.

It's minimalist and unified API design can greatly reduce the learning costs of new users. With it, you can better focus on the data itself, rather than the complicated web crawler design. If you are familiar with java and hoping to crawl some interesting data, just hold on, you will soon carry out your first crawler in few minutes. Ok, let's start!
It's minimalist and unified API design can greatly reduce the learning costs of new users. With it, you can better focus on the data itself, rather than design the complicated web crawler from the beginning. If you are familiar with java and hoping to crawl some interesting data, just hold on, you will soon carry out your first crawler in few minutes. Ok, let's start!

## Features

Expand Down Expand Up @@ -44,7 +44,7 @@ Module division:

![](http://7xp1jv.com1.z0.glb.clouddn.com/gospy/img/single-infra.jpg?imageView/5/w/500)

Cluster structure:
Run in cluster:

## Quick start

Expand Down
169 changes: 151 additions & 18 deletions pom.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,32 +7,165 @@

<groupId>cc.gospy</groupId>
<artifactId>gospy-core</artifactId>
<packaging>jar</packaging>
<name>gospy-core</name>
<version>0.2.1</version>
<description>scalable web crawler framework</description>
<url>http://www.gospy.cc/</url>

<name>gospy-core</name>
<url>www.gospy.cc</url>
<licenses>
<license>
<name>The Apache Software License, Version 2.0</name>
<url>http://www.apache.org/licenses/LICENSE-2.0.txt</url>
</license>
</licenses>

<developers>
<developer>
<name>Zhang Jiupeng</name>
<email>[email protected]</email>
<url>https://www.zhangjiupeng.com/</url>
</developer>
</developers>

<scm>
<connection>scm:[email protected]:zhangjiupeng/gospy.git</connection>
<developerConnection>scm:[email protected]:zhangjiupeng/gospy.git</developerConnection>
<url>[email protected]:zhangjiupeng/gospy.git</url>
</scm>

<profiles>
<profile>
<id>release</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-source-plugin</artifactId>
<version>3.0.1</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>jar-no-fork</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<version>2.10.4</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>jar</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-gpg-plugin</artifactId>
<version>1.6</version>
<executions>
<execution>
<id>sign-artifacts</id>
<phase>verify</phase>
<goals>
<goal>sign</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>

<properties>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>

<build>
<resources>
<resource>
<directory>src/main/resources</directory>
<excludes>
<exclude>**/*.log</exclude>
<exclude>**/*.json</exclude>
</excludes>
<includes>
<!--<include>**/*.xml</include>-->
<include>**/*.html</include>
<include>**/*.js</include>
</includes>
<filtering>false</filtering>
</resource>
</resources>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.3.2</version>
<version>3.6.0</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
<verbose>true</verbose>
<fork>true</fork>
<encoding>UTF-8</encoding>
</configuration>
</plugin>
</plugins>
</pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.8</source>
<target>1.8</target>
<compilerVersion>1.8</compilerVersion>
</configuration>
</plugin>

<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>3.0.0</version>
<executions>
<execution>
<id>copy-dependencies</id>
<phase>package</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<outputDirectory>${project.build.directory}/lib</outputDirectory>
<overWriteReleases>false</overWriteReleases>
<overWriteSnapshots>false</overWriteSnapshots>
<overWriteIfNewer>true</overWriteIfNewer>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>

<distributionManagement>
<snapshotRepository>
<id>oss</id>
<url>https://oss.sonatype.org/content/repositories/snapshots/</url>
</snapshotRepository>
<repository>
<id>oss</id>
<url>https://oss.sonatype.org/service/local/staging/deploy/maven2/</url>
</repository>
</distributionManagement>

<dependencies>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
Expand Down Expand Up @@ -127,20 +260,20 @@
<version>3.8.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>cc.agentx</groupId>
<artifactId>agentx</artifactId>
<version>1.2.1</version>
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</exclusion>
<exclusion>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
</exclusion>
</exclusions>
</dependency>
<!--<dependency>-->
<!--<groupId>cc.agentx</groupId>-->
<!--<artifactId>agentx</artifactId>-->
<!--<version>1.2.1</version>-->
<!--<exclusions>-->
<!--<exclusion>-->
<!--<groupId>org.slf4j</groupId>-->
<!--<artifactId>slf4j-api</artifactId>-->
<!--</exclusion>-->
<!--<exclusion>-->
<!--<groupId>ch.qos.logback</groupId>-->
<!--<artifactId>logback-classic</artifactId>-->
<!--</exclusion>-->
<!--</exclusions>-->
<!--</dependency>-->
</dependencies>
</project>
60 changes: 0 additions & 60 deletions src/main/java/cc/gospy/example/func/TinyDemo.java
View file

This file was deleted.

0 comments on commit 37a2fff

Please sign in to comment.