Compare commits

...

142 Commits

Author SHA1 Message Date
lionsoul 41b9c2d763 Track the CHANGES for version 1.8.0 2018-12-31 13:24:03 +08:00
lionsoul 74b2909ea2 ip source text and binary data upgrade to version 2018/12/31 2018-12-31 13:03:50 +08:00
lionsoul ab1c331fbd Update the merge to version 1.2.1 2018-11-03 11:05:48 +08:00
lionsoul a6968d9767
Merge pull request #86 from heweizhen0822/patch-1
Update README.md
2018-10-18 20:59:29 +08:00
lionsoul e4d3772e08
Merge pull request #85 from heweizhen0822/patch-2
Update main.go
2018-10-18 20:59:09 +08:00
heweizhen0822 06e8aa2528
Update main.go
github.com/mohong122/ip2region/binding/golang/ip2region 并没有子包ip2region 需要改成 github.com/lionsoul2014/ip2region/binding/golang/ip2region
2018-10-18 10:34:33 +08:00
heweizhen0822 281237c5c3
Update README.md
github.com/mohong122/ip2region/binding/golang/ip2region  需要改成 github.com/lionsoul2014/ip2region/binding/golang/ip2region
2018-10-18 10:32:32 +08:00
lionsoul 9c37b5be7f Optimize the lua_c module 2018-10-10 11:54:26 +08:00
lionsoul 253dc12917 Add automatically lib dir make 2018-10-07 20:58:54 +08:00
lionsoul 86104eb31f Optimize the ip2long function with trim logic applied 2018-10-07 10:23:17 +08:00
lionsoul bd6dc7be90 Update the README doc for lua_c module 2018-10-06 15:12:24 +08:00
lionsoul 03fd40bb09 Add README doc for lua_c module 2018-10-06 11:18:01 +08:00
lionsoul f9e9dd59a3 lua ip2region c module forward with release version published 2018-10-06 11:05:01 +08:00
lionsoul 93689e09e8 lua binding module interface stdlize with new and ip2long method access by . support 2018-10-05 21:46:21 +08:00
lionsoul e26ed44861 lua_c module forward with basic framework finished(tested) 2018-10-05 21:39:35 +08:00
lionsoul 7bd81290f1 lua_c binding basic framework(Not tested yet...) 2018-10-05 14:03:16 +08:00
lionsoul 7032830d23 Optimize the lua module by add the __index and the __tostring meta-method 2018-10-05 10:55:24 +08:00
lionsoul e9a08e27a3 Optimize the benchmark script 2018-10-05 09:49:32 +08:00
lionsoul 8e278d9c25
Merge pull request #81 from KomaBeyond/master
重构python客户端,提升查询性能,兼容python2.x,3.x
2018-10-05 09:40:15 +08:00
Koma 593aba66d8 重构python客户端,提升查询性能,兼容python2.x,3.x 2018-10-04 11:39:49 +08:00
lionsoul a2d250919d Update the lua interface and add setDbFile interface 2018-10-04 10:22:01 +08:00
lionsoul 5f871ba257 Update the Lua binding 2018-10-03 20:12:30 +08:00
lionsoul e6104dcd1f Update README for lua binding 2018-10-03 20:10:02 +08:00
lionsoul 86ad8057aa Lua binding completed(tested) 2018-10-03 20:05:14 +08:00
lionsoul 5f073d814e lua binding forward with binarySearch and btreeSearch interface implemented 2018-10-03 13:12:05 +08:00
lionsoul 04f093d9bd lua binding basic framework with memory search completed(tested) 2018-10-02 22:36:57 +08:00
lionsoul 1f04c554db Update the maven version number to 1.7.2 2018-10-02 11:24:58 +08:00
lionsoul 679f30acdb Add DbSearcher(dbConfig, dbBinStr) construct method specified by Wendal at https://gitee.com/lionsoul/ip2region/issues/IILFL 2018-10-02 11:14:31 +08:00
lionsoul cd84dff3ac Add some notices 2018-10-02 10:39:54 +08:00
lionsoul d82447c1b7
Merge pull request #76 from a631807682/master
[nodejs] publish package on npm
2018-10-02 10:19:43 +08:00
lionsoul 7322733691 Merge branch 'master' of https://github.com/lionsoul2014/ip2region 2018-09-05 20:45:30 +08:00
lionsoul 1312bdba32 Update the maven version to 1.7 2018-09-05 20:44:37 +08:00
lionsoul 9623e345d9
Merge pull request #72 from stulzq/master
Add use stream method for C#
2018-09-01 20:59:52 +08:00
陈睿 4556afd815 [nodejs] publish package on npm 2018-08-30 18:20:49 +08:00
lionsoul f482591eee
Merge pull request #69 from Maledong/master
[Feature] Add 'memorySearch' and 'memorySearchSync' for Nodejs
2018-08-25 10:59:30 +08:00
stulzq f3ae83afb5 add use stream method 2018-08-19 16:47:01 +08:00
MaleDong 487129d59c [Feature] Add 'memorySearch' and 'memorySearchSync' for Nodejs
1) Add features.
2) Add unit tests.
3) Add benchmark.
2018-07-31 16:12:56 +08:00
lionsoul b7e9755c6b java binding version 1.7 with btree search bug fixed 2018-07-31 16:03:15 +08:00
lionsoul f82d3be466 Fix the bug for btree header fetch 2018-07-31 15:58:11 +08:00
lionsoul f2f80fc5b3 Update to the latest data of version 2018/07/26 2018-07-31 15:49:28 +08:00
lionsoul 4feeb03f85
Merge pull request #68 from RocherKong/master
Use the synchronization method as the first choice
2018-07-31 12:43:11 +08:00
Rocher e1a7052c9d
Update README.md
Use the synchronization method as the first choice.
2018-07-31 11:37:25 +08:00
Rocher 48edca6313
Update README.md
Use the synchronization method as the first choice.
2018-07-31 11:36:14 +08:00
Rocher c4583dc6fd
Merge pull request #3 from lionsoul2014/master
pull newest
2018-07-31 11:33:02 +08:00
lionsoul 6cd49f341d
Merge pull request #67 from Maledong/master
[Fix] Fix csharp formations
2018-07-30 15:26:37 +08:00
MaleDong b177834c7b [Fix] Fix csharp formations
1) In `README.md`, we shouldn't use `c#` but `csharp` to make md
recoginize it's the C# formation of codes.

2) Add 'ConfigureAwait(false)' as for the MSDN magazine usage.

3) Don't need Contribution History in README.md, because we can see the
change logs in GitHub instead.
2018-07-30 14:44:47 +08:00
lionsoul 1cde91a274
Merge pull request #66 from Maledong/NeedDispose
[Fix] Add "Disposible" interface for final release of resouces
2018-07-30 11:31:18 +08:00
MaleDong ee5c05d321 [Fix] Add "Disposible" interface for final release of resouces
`_search` should release resouces because the file state is still open.
2018-07-30 11:21:31 +08:00
lionsoul 3cf1c78ff1
Merge pull request #65 from Maledong/ChangeToTask
[Update] Refactor codes according to RocherHong's changes
2018-07-30 10:56:10 +08:00
MaleDong c6721af1ec [Update] Refactor codes according to RocherHong's changes
1) Considering using xUnit for UnitTest, remove my unit tests.
2) Remove useless `async`, because `async` is just the wrapper of `Task`.
3) Modify unit tests to be sure that the values MUST BE equal to each other when the IP is the same.
2018-07-30 10:48:30 +08:00
lionsoul d40e001802
Merge pull request #61 from Maledong/master
New Feature of async for Nodejs, Refactor of codes in node.js
2018-07-30 10:41:23 +08:00
lionsoul d51db48721 Fix the conflicts 2018-07-30 10:39:15 +08:00
lionsoul a1e433d9c0
Merge pull request #63 from RocherKong/master
Code Refactoring
2018-07-30 10:26:21 +08:00
MaleDong 8a6ac1cdc6 Change from `async Task` to `Task`
Because the async is the wrapper of Task, we don't need to use async
again.
2018-07-30 09:56:25 +08:00
RocherKong 55db35b6f4 1.CodeStandardized 2.Support Netfx4.5 3.TestStandardized 2018-07-30 00:31:49 +08:00
Rocher f0c3b88653
Merge pull request #2 from lionsoul2014/master
merge newest
2018-07-30 00:27:46 +08:00
MaleDong 1331f75997 Refactor of codes in node.js
1) Add async method: `binarySearch` and `btreeSearch`.
2) Refactor of tests by putting them into `tests` folder.
3) Create several new tests in details.
4) Remove useless and reformat codes to be clear.
5) Remove useless snapshots, because they can be recreated when you run `npm run test`.
2018-07-29 20:02:14 +08:00
lionsoul 57d9cd788f
Merge pull request #62 from Maledong/RefactorCSharp
Refactor of CSharp
2018-07-28 15:32:50 +08:00
MaleDong 33570612bc Refactor of CSharp:
1) Add `Async` methods.
2) Remove uesless variables, codes.
3) Remove ConsoleApp and use UnitTest instead.
4) Add README.md instead.
2018-07-20 13:57:32 +08:00
Rocher 7a9e9a6504
Merge pull request #1 from lionsoul2014/master
update to newest
2018-07-17 15:32:37 +08:00
lionsoul 71ec923046
Merge pull request #57 from biluohc/master
Rust: 0.2.0, use std::net::Ipaddr and update Error.
2018-07-03 20:04:22 +08:00
biluohc ea9a3ff7b8 Rust: 0.2.0, use std::net::Ipaddr and update Error. 2018-07-03 15:53:16 +08:00
dongyado 94ed13cdd0 update php_extension client intro file 2018-07-03 15:27:33 +08:00
dongyado b63a566641 add new author and email for nodejs binding client 2018-07-03 15:17:47 +08:00
Monster Slayer 1d3c083dc8
Merge pull request #50 from leeching/refactor/nodejs
refactor(nodejs): write nodejs script in javascript way
2018-07-03 15:14:41 +08:00
lionsoul 4afaf4513c
Merge pull request #55 from biluohc/master
add rust implement.
2018-07-03 10:55:40 +08:00
leeching 1ffe2b985e feat: add destroy method for backward compatibility 2018-07-03 10:07:25 +08:00
leeching 834e5939d8 feat(nodejs): add create static method 2018-07-03 09:48:41 +08:00
leeching b89ed1c397 refactor(nodejs): write nodejs script in javascript way 2018-07-03 09:48:41 +08:00
biluohc 1d09fee3c5 add a example 2018-07-02 23:04:46 +08:00
biluohc 02db5c5c56 use workspace 2018-07-02 22:17:10 +08:00
biluohc 4120c651eb add rust implement. 2018-06-30 16:16:00 +08:00
lionsoul 751c095ca0
Merge pull request #54 from Insua/master
fix issuse golang执行测试报错 #29
2018-06-30 10:40:37 +08:00
insua d8a7e34a84 fix(move golang package in directory and typo in readme): 2018-06-29 16:49:38 +08:00
lionsoul b6da4602e5
Merge pull request #52 from felixonmars/patch-1
Fix some typos in Ip2Region.class.php
2018-06-28 10:51:19 +08:00
Felix Yan 34be04bc4a
Fix some typos in Ip2Region.class.php 2018-06-28 02:00:05 +08:00
dongyado 550425a2b5 keep php class and php_extention return data key same, we change php_extension return data key : cityId to city_id 2018-06-27 10:44:29 +08:00
lionsoul 35cc0f37c7
Merge pull request #48 from leeching/fix/nodejs-var
remove unnecessary var
2018-06-26 17:34:03 +08:00
lionsoul 250b99de29
Merge pull request #49 from leeching/fix/nodejs-snapshot
test(nodejs): update snapshot
2018-06-26 17:33:08 +08:00
leeching b961c1f0ac test(nodejs): update snapshot 2018-06-26 17:10:56 +08:00
leeching 3d6db4931a remove unnecessary var 2018-06-26 16:26:00 +08:00
lionsoul 1634e7ae5e
Merge pull request #47 from jwcn/master
重命名 php 测试脚本
2018-06-26 12:37:00 +08:00
Wei.Jiang 470b29db3c
Merge pull request #1 from jwcn/jwcn-patch-1
重命名 php 测试脚本
2018-06-26 11:06:46 +08:00
Wei.Jiang 98616d0df8
重命名 php 测试脚本
重命名 php 测试脚本,与 README 和其他语言脚本名保持一致
2018-06-26 11:05:48 +08:00
lionsoul 1b275b49da
Merge pull request #46 from WoShiSunWuKong/patch-1
add nodejs sample
2018-06-26 10:09:09 +08:00
我是孙悟空 fb732f0535
add nodejs sampale 2018-06-25 12:59:52 +08:00
lionsoul 02506277ad fix the region and re-generate the ip2region.db file 2018-06-19 17:38:33 +08:00
dongyado 7e6cbdc652 update php_extension install guides 2018-05-17 12:11:13 +08:00
lionsoul 0cab5514eb
Merge pull request #44 from Chunlin-Li/fix-binding-nodejs
fix: ReferenceError: m is not defined
2018-05-17 09:20:26 +08:00
Chunlin Li fdac1c0101
fix: ReferenceError: m is not defined 2018-05-16 19:01:17 +08:00
lionsoul 42764c495b Ignore the testSearch generated by Makefile 2018-04-09 09:31:48 +08:00
lionsoul 09c2ba55fc
Merge pull request #41 from helloKaiDing/master
Makefile for binding/c with ip2region.db auto generate.
2018-04-09 09:28:45 +08:00
helloKaiDing d4dab91796
Makefile for binding/c
This is an Makefile for the binding/c program, can use to generate the db file ip2config.db  and the test program testSearcher, can be very convient for the new starter.
2018-04-08 17:23:03 +08:00
lionsoul d22d96cae2
Merge pull request #39 from panjunjie/patch-1
Update README.md
2018-03-30 00:39:55 +08:00
Jack a1ae4c8299
Update README.md
更改 markdow h 标签错误格式!
2018-03-29 19:04:48 +08:00
lionsoul 74b3045339 Update the ip data to version 2018/03/26 2018-03-27 14:38:33 +08:00
lionsoul 7cb8bc4b9d
Merge pull request #31 from RocherKong/master
add C# support ( from RocherKong)
2018-02-11 11:56:51 +08:00
RocherKong b65d8206ba add support for C# 2018-02-09 22:04:01 +08:00
lionsoul 39555fc266 Update the data to version 2018/01/12 2018-01-16 15:08:30 +08:00
lionsoul 68540b4db4 Fix the doc error 2017-11-23 21:25:32 +08:00
lionsoul 217a7dab9e Untrack the binding/java/target/ directory 2017-10-28 15:38:09 +08:00
lionsoul 07ea5994b8 Update the REAMD 2017-10-23 15:31:46 +08:00
lionsoul 02391e2057 Add the maven repository address 2017-10-23 13:50:57 +08:00
lionsoul 5f29715ec2 java binding maven upload & track the latest dbMaker.jar 2017-10-23 13:48:49 +08:00
lionsoul 4d0ce55008 bold the upgrade notice 2017-10-21 13:10:16 +08:00
lionsoul 29bacb0c8a Update the README 2017-10-21 13:08:26 +08:00
lionsoul a49fae2ce8 Update the README 2017-10-21 13:07:29 +08:00
lionsoul 1f0b07d42b Update the notice 2017-10-21 13:06:08 +08:00
lionsoul b0aecd7046 active the upgrade notice 2017-10-21 13:05:14 +08:00
lionsoul b48dbe5248 Bug fixed: Adjust the super header length to 8192 for all client implementation 2017-10-21 12:47:24 +08:00
Wilson Wang 4ffb1334a1 !4 python 3正常运行
Merge pull request !4 from Wilson Wang/master
2017-10-21 12:13:29 +08:00
wfifi 0f39924b08 add python3 support 2017-10-20 18:09:07 +08:00
lionsoul 7f1216eb6f Merge branch 'master' of http://git.oschina.net/lionsoul/ip2region 2017-10-20 12:13:30 +08:00
lionsoul 6eec7a2c42 Update the to data 2017/10/09 version 2017-10-20 12:13:19 +08:00
lionsoul 499bc1887e Merge pull request #19 from chengpiaopiao/master
fix incorrect python test helper.
2017-09-16 09:39:27 +08:00
chengpiaopiao 1ad6274118 fix incorrect python test helper. 2017-09-15 15:40:04 +08:00
dongyado 769ab315b1 correct post path 2017-09-08 14:10:31 +08:00
lionsoul 979e9bdda7 Merge pull request #15 from mohong122/master
Golang binding 增加控制台实例
2017-08-01 12:18:56 +08:00
mohong e4f54054c4 Merge pull request #1 from lionsoul2014/master
合并远程
2017-05-25 14:51:46 +08:00
lionsoul af9c8794ff Merge branch 'master' of https://github.com/lionsoul2014/ip2region 2017-04-19 10:49:13 +08:00
lionsoul f93b8d2dfb Fix the demo doc error 2017-04-19 10:48:10 +08:00
lionsoul 546be60b0a Merge pull request #11 from zhuangya/jest
chore: test in jest
2017-04-14 13:11:52 +08:00
Ya Zhuang eae269f37d
chore: test in jest 2017-04-14 12:47:55 +08:00
lionsoul 90682e06eb track the latest binary db data 2017-04-10 11:04:02 +08:00
lionsoul 7d4a36d9bc Update the 1.2.4 version desc 2017-04-07 19:04:12 +08:00
lionsoul fe4057d8aa Add the changes for version 1.2.4 2017-04-07 18:53:55 +08:00
lionsoul bd3ffadabf Update the source and the binary ip data to version 2017/03/15 2017-04-07 18:53:29 +08:00
lionsoul b7376d5868 Add the author info for binding/c_mmap module 2017-01-12 18:21:36 +08:00
lionsoul 387f5a2097 Merge branch 'master' of http://git.oschina.net/lionsoul/ip2region 2017-01-12 18:14:16 +08:00
lionsoul 9de920ef2b stdlize the code style and add the author info 2017-01-12 18:14:09 +08:00
我的上铺叫路遥 639c852534 Merge pull request !3 from 我的上铺叫路遥/master 2017-01-12 18:13:16 +08:00
Leo Ma 3c8cfa1282 Fix memory leak
Signed-off-by: Leo Ma <begeekmyfriend@gmail.com>
2017-01-12 18:04:07 +08:00
我的上铺叫路遥 a67936d835 Merge pull request !2 from 我的上铺叫路遥/master 2017-01-12 17:51:43 +08:00
Leo Ma 9bdf030fe0 Add database file memory map as c_mmap binding
File I/O optimization.

- No more `fread` call when the file is mapped.

- Close database file as soon as possible after `mmap`

Signed-off-by: Leo Ma <begeekmyfriend@gmail.com>
2017-01-12 17:33:58 +08:00
lionsoul 7a2b9943b5 stdlize the license add free statement 2016-12-28 09:53:13 +08:00
lionsoul c0955fedff update the merge&binary db file to version/2016/12/14 2016-12-21 16:53:14 +08:00
lionsoul 21bb060839 fix 10+ rows failed ip range records 2016-11-07 14:07:03 +08:00
lionsoul b0f7bff2de text and binary data version 2016/11/02 2016-11-05 21:13:36 +08:00
lionsoul 643e6a2463 update the db make doc and jar file 2016-11-03 17:32:16 +08:00
lionsoul ec395be524 add db make doc & update the CHANGES 2016-11-03 16:40:28 +08:00
lionsoul 7dcec80080 track the db make jar tools 2016-11-03 15:53:20 +08:00
lionsoul bedaca5b70 Merge pull request #4 from mohong122/master
golang实现
2016-10-20 17:08:17 +08:00
94 changed files with 505431 additions and 83074 deletions

32
.gitignore vendored
View File

@ -3,12 +3,44 @@
*.o *.o
*.pyc *.pyc
*~ *~
*.log
*.la
*.so
META-INF/ META-INF/
# Binary Files # # Binary Files #
*.jar *.jar
!dbMaker-*.jar
# Package Files # # Package Files #
.settings/ .settings/
.classpath .classpath
.project .project
# vim swp file # # vim swp file #
*.swp *.swp
.idea
# binding
/binding/java/classes/
/binding/java/doc/
/binding/java/target/
/binding/java/*.jar
/binding/c/testSearcher
# rust
Cargo.lock
target
# VS ignore cases
/**/*.sln
/binding/c#/**/.vs/
/binding/c#/**/packages
/binding/c#/**/bin
/binding/c#/**/obj
# Nodejs
/binding/nodejs/tests/unitTests/__snapshots__
/binding/nodejs/coverage

View File

@ -1,3 +1,47 @@
### 1.8
1. 数据升级至2018/12/31的版本国外的也增加了大量的城市数据。
2. Java的binding的Searcher类增加了如下的接口和实现
```
DbSearcher(DbConfig dbConfig, string dbBinStr);
```
3. Javascript的pacakge上传到了npm。
4. 增加纯lua实现的lua binding。
5. 增加C实现的lua_c binding。
6. 优化部分binding的demo运行和使用文档。
7. 修复python binding的benchmark的部分错误。
### 1.2.4
1, 花了近两周的时间重写了数据的升级算法,再次提高准确率,升级过程如下(此处应该有掌声):
```shell
1. ip预分段利于分布式同步升级
2. 分段同步升级,目前使用四个机器同时升级
3自动验证数据查缺补全
4数据合并
5数据格式标准化目前2个工作
1)ipip.net的数据和淘宝数据保持区域名称统一ipip.net的数据不带“省”和“市”关键字。
2)香港澳门台湾等国家信息修复淘宝原始数据有问题reported at http://git.oschina.net/lionsoul/ip2region/issues/21
6重复ip段的合并得到data目录下的ip.merge.txt
7生产data目录下的ip2region.db二进制数据库文件
```
2修复原始数据关于“香港台湾澳门”国家信息错误的bugreported at http://git.oschina.net/lionsoul/ip2region/issues/21
3将数据升级至2017/03/15同步版本
4增加c_mmap查询客户端C客户端查询文件读取使用内存映射加速感谢[Leo Ma](http://git.oschina.net/begeekmyfriend)的贡献
备注部分阿里云数据有问题缺失的部分使用免费开放的ipip.net的数据代替了。
### 1.2.3
1. 优化升级算法进行更深度的分裂可以使准确率再度提高同时数据的更新也花费了2天多的时间
2. 数据升级至2016/12/14版本的数据包括原始数据和binary数据库的数据
### 1.2.2
1. ip数据文件更新至2016/11/02版本
2. 增加ip2region.db数据库文件的java生成工具请参考README获取使用方式
3. 新增golang查询客户端的实现感谢 [@mohong122](https://github.com/mohong122) 的关注和贡献
### 1.2 ### 1.2
1. 新增分列式升级算法大大的增加了数据的准确率基本避免之前各大网友反馈的些许ip定位错误 1. 新增分列式升级算法大大的增加了数据的准确率基本避免之前各大网友反馈的些许ip定位错误

2
Cargo.toml Normal file
View File

@ -0,0 +1,2 @@
[workspace]
members = ["binding/rust", "binding/rust/example"]

View File

@ -201,9 +201,9 @@
limitations under the License. limitations under the License.
========================================================================== ==========================================================================
The following license applies to the JQuery JavaScript library The following license applies to the ip2region library
-------------------------------------------------------------------------- --------------------------------------------------------------------------
Copyright (c) 2010 John Resig, http://jquery.com/ Copyright (c) 2015 Lionsoul<chenxin619315@gmail.com>
Permission is hereby granted, free of charge, to any person obtaining Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the a copy of this software and associated documentation files (the
@ -223,4 +223,3 @@ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

View File

@ -1,57 +1,85 @@
ip2region - 最自由的ip地址查询库ip到地区的映射库提供Binary,B树和纯内存三种查询算法妈妈再也不用担心我的ip地址定位。 ip2region - 最自由的ip地址查询库ip到地区的映射库提供Binary,B树和纯内存三种查询算法妈妈再也不用担心我的ip地址定位。
**1. 99.9%准确率,定时更新:** ### 1. 99.9%准确率,定时更新:
数据聚合了一些知名ip到地名查询提供商的数据这些是他们官方的的准确率经测试着实比纯真啥的准确多了。<br /> 数据聚合了一些知名ip到地名查询提供商的数据这些是他们官方的的准确率经测试着实比纯真啥的准确多了。<br />
每次聚合一下数据需要1-2天会不定时更新。 每次聚合一下数据需要1-2天会不定时更新。
**2. 标准化的数据格式:** ### 2. 标准化的数据格式:
每条ip数据段都固定了格式_城市Id|国家|区域|省份|城市|ISP_ 每条ip数据段都固定了格式_城市Id|国家|区域|省份|城市|ISP_
只有中国的数据精确到了城市其他国家只能定位到国家后前的选项全部是0已经包含了全部你能查到的大大小小的国家。 只有中国的数据精确到了城市其他国家只能定位到国家后前的选项全部是0已经包含了全部你能查到的大大小小的国家。
请忽略前面的城市Id个人项目需求 请忽略前面的城市Id个人项目需求
**3. 体积小:** ### 3. 体积小:
生成的数据库文件ip2region.db只有1.5M1.2版本前是3.5M 生成的数据库文件ip2region.db只有1.5M1.2版本前是3.5M
**4. 多查询客户端的支持0.0x毫秒级别的查询** ### 4. 多查询客户端的支持0.0x毫秒级别的查询
已经集成的客户端有java, php, c, pythonnodejsphp扩展(支持linux, php5, php7版本已支持) 已经集成的客户端有java、C#、php、c、python、nodejs、php扩展(php5和php7)、golang、rust、lua、lua_c
提供了两种查询算法,响应时间如下: 提供了两种查询算法,响应时间如下:
客户端/binary算法/b-tree算法/Memory算法 客户端/binary算法/b-tree算法/Memory算法
c#/0.x毫秒/0.x毫秒/0.x毫秒
java/0.x毫秒/0.x毫秒/0.1x毫秒 (使用RandomAccessFile) java/0.x毫秒/0.x毫秒/0.1x毫秒 (使用RandomAccessFile)
php/0.x毫秒/0.1x毫秒/0.1x毫秒 php/0.x毫秒/0.1x毫秒/0.1x毫秒
c/0.0x毫秒/0.0x毫秒/0.00x毫秒(b-tree算法基本稳定在0.02x毫秒级别) c/0.0x毫秒/0.0x毫秒/0.00x毫秒(b-tree算法基本稳定在0.02x毫秒级别)
python/0.x毫秒/0.1x毫秒/未知 python/0.x毫秒/0.1x毫秒/未知
lua/0.x毫秒/0.x毫秒/0.x毫秒
lua_c/0.0x毫秒/0.0x毫秒/0.00x毫秒
任何客户端b-tree都比binary算法快当然Memory算法固然是最快的 任何客户端b-tree都比binary算法快当然Memory算法固然是最快的
**5. 测试程序:** maven仓库地址
java: ```xml
<dependency>
<groupId>org.lionsoul</groupId>
<artifactId>ip2region</artifactId>
<version>1.7.2</version>
</dependency>
```
nuget安装命令
```shell
Install-Package IP2Region
```
### 5. 测试程序:
C#、Node.js、lua_c的测试请具体参考文件夹中README.md 说明。
java:
```shell
cd binding/java cd binding/java
ant all ant all
java -jar ip2region-{version}.jar ./data/ip2region.db java -jar ip2region-{version}.jar ./data/ip2region.db
```
php: php:
```shell
php binding/php/testSearcher ./data/ip2region.db php binding/php/testSearcher.php ./data/ip2region.db
```
c: c:
```shell
cd binding/c/ cd binding/c/
gcc -g -O2 testSearcher.c ip2region.c gcc -g -O2 testSearcher.c ip2region.c
./a.out ../../data/ip2region.db ./a.out ../../data/ip2region.db
```
python: python:
```shell
python binding/python/testSearcher.py ./data/ip2region.db
```
python binding/python/testSearcher ./data/ip2region.db lua:
```shell
cd binding/lua/
lua testSearcher.lua ../../data/ip2region.db
```
均会看到如下界面: 均会看到如下界面:
```shell
initializing B-tree ... initializing B-tree ...
+----------------------------------+ +----------------------------------+
| ip2region test script | | ip2region test script |
@ -60,7 +88,41 @@ python:
+----------------------------------+ +----------------------------------+
p2region>> 101.105.35.57 p2region>> 101.105.35.57
2163|中国|华南|广东省|深圳市|鹏博士 in 0.02295 millseconds 2163|中国|华南|广东省|深圳市|鹏博士 in 0.02295 millseconds
```
输入ip地址开始测试第一次会稍微有点慢在运行命令后面接入binary,memory来尝试其他算法建议使用b-tree算法速度和并发需求的可以使用memory算法。 输入ip地址开始测试第一次会稍微有点慢在运行命令后面接入binary,memory来尝试其他算法建议使用b-tree算法速度和并发需求的可以使用memory算法。
具体集成请参考不同客户端的测试源码。 具体集成请参考不同客户端的测试源码。
### 6. 如何生成ip2region.db文件
从ip2region 1.2.2版本开始里面提交了一个dbMaker-{version}.jar的可以执行jar文件用它来完成这个工作
* 1, 确保你安装好了java环境不玩Java的童鞋就自己谷歌找找拉临时用一用几分钟的事情
* 2, cd到ip2region的根目录然后运行如下命令
```shell
java -jar dbMaker-{version}.jar -src 文本数据文件 -region 地域csv文件 [-dst 生成的ip2region.db文件的目录]
# 文本数据文件db文件的原始文本数据文件路径自带的ip2region.db文件就是/data/ip.merge.txt生成而来的你可以换成自己的或者更改/data/ip.merge.txt重新生成
# 地域csv文件该文件目的是方便配置ip2region进行数据关系的存储得到的数据包含一个city_id这个直接使用/data/origin/global_region.csv文件即可
# ip2region.db文件的目录是可选参数没有指定的话会在当前目录生成一份./data/ip2region.db文件
```
* 3, 获取生成的ip2region.db文件覆盖原来的ip2region.db文件即可
* 4, 默认的ip2region.db文件生成命令:
```shell
cd ip2region项目根目录
java -jar dbMaker-1.2.2.jar -src ./data/ip.merge.txt -region ./data/global_region.csv
# 会看到一大片的输出
```
* 5, 数据库文件的结构和原理请阅读 @冬芽 的blog[“ip2region数据库文件的结构和原理”](http://dongyado.com/tool/2016/08/18/structure-of-ip2region-database-file/)
### 7. 其他备注
* 1, 全部binding的各个search接口都<b>不是</b>线程安全的实现,不同线程可以通过创建不同的查询对象来使用。
* 2, memorySearch接口在发布对象前进行一次预查询可以安全用于多线程环境。
* 3, ip2region交流分享微信lionsoul2014QQ1187582057

View File

@ -0,0 +1,54 @@
using BenchmarkDotNet.Attributes;
using IP2Region.Models;
using System.Threading.Tasks;
namespace IP2Region.Test.Benchmark
{
[RankColumn]
public class DbSearch_Test : TestBase
{
private string RandomIP = "";
[Benchmark]
public DataBlock MemorySearch()
{
RandomIP = GetRandomIP();
return _search.MemorySearch(RandomIP);
}
[Benchmark]
public async Task<DataBlock> MemorySearch_Async()
{
RandomIP = GetRandomIP();
return await _search.MemorySearchAsync(RandomIP);
}
[Benchmark]
public DataBlock BinarySearch()
{
RandomIP = GetRandomIP();
return _search.BinarySearch(RandomIP);
}
[Benchmark]
public async Task<DataBlock> BinarySearch_Async()
{
RandomIP = GetRandomIP();
return await _search.BinarySearchAsync(RandomIP);
}
[Benchmark]
public DataBlock BtreeSearch()
{
RandomIP = GetRandomIP();
return _search.BtreeSearch(RandomIP);
}
[Benchmark]
public async Task<DataBlock> BtreeSearch_Async()
{
RandomIP = GetRandomIP();
return await _search.BtreeSearchAsync(RandomIP);
}
}
}

View File

@ -0,0 +1,16 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp2.0</TargetFramework>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="BenchmarkDotNet" Version="0.11.0" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\IP2Region\IP2Region.csproj" />
</ItemGroup>
</Project>

View File

@ -0,0 +1,15 @@
using BenchmarkDotNet.Running;
using System;
namespace IP2Region.Test.Benchmark
{
class Program
{
static void Main(string[] args)
{
Console.WriteLine("Now starting benchmark test, please wait......");
var summary = BenchmarkRunner.Run<DbSearch_Test>();
Console.ReadLine();
}
}
}

View File

@ -0,0 +1,49 @@
using BenchmarkDotNet.Attributes;
using System;
namespace IP2Region.Test.Benchmark
{
public class TestBase
{
protected DbSearcher _search;
private readonly string _dBFilePath = "";
public TestBase()
{
}
public TestBase(String DBFilePath)
{
_dBFilePath = DBFilePath;
}
[GlobalSetup]
public void Init()
{
if (String.IsNullOrEmpty(_dBFilePath))
{
_search = new DbSearcher(AppContext.BaseDirectory + @"\DB\ip2region.db");
}
else
{
_search = new DbSearcher(_dBFilePath);
}
}
[GlobalCleanup]
public void Dispose()
{
_search.Dispose();
}
public String GetRandomIP()
{
return new Random(Guid.NewGuid().GetHashCode()).Next(0, 255).ToString() + "."
+ new Random(Guid.NewGuid().GetHashCode()).Next(0, 255).ToString() + "."
+ new Random(Guid.NewGuid().GetHashCode()).Next(0, 255).ToString() + "."
+ new Random(Guid.NewGuid().GetHashCode()).Next(0, 255).ToString();
}
}
}

Binary file not shown.

View File

@ -0,0 +1,26 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netcoreapp2.0</TargetFramework>
<IsPackable>false</IsPackable>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.7.0" />
<PackageReference Include="xunit" Version="2.3.1" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.3.1" />
<DotNetCliToolReference Include="dotnet-xunit" Version="2.3.1" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\IP2Region\IP2Region.csproj" />
</ItemGroup>
<ItemGroup>
<None Update="DB\ip2region.db">
<CopyToOutputDirectory>Always</CopyToOutputDirectory>
</None>
</ItemGroup>
</Project>

View File

@ -0,0 +1,51 @@
using System;
using System.Threading.Tasks;
using Xunit;
namespace IP2Region.Test.xUnit
{
public class SearchTest : IDisposable
{
private readonly DbSearcher _search;
public SearchTest()
{
_search = new DbSearcher(Environment.CurrentDirectory + @"\DB\ip2region.db");
}
[Fact]
public void Search_Test()
{
string memResult = _search.MemorySearch("183.192.62.65").Region;
string binarySearchResult = _search.BinarySearch("183.192.62.65").Region;
string binaryTreeSearchResult = _search.BtreeSearch("183.192.62.65").Region;
Assert.NotNull(memResult);
Assert.NotNull(binarySearchResult);
Assert.NotNull(binaryTreeSearchResult);
Assert.Equal(memResult, binarySearchResult);
Assert.Equal(binaryTreeSearchResult, memResult);
}
[Fact]
public async Task SearchAsync_Test()
{
// We don't need the synchronizeContext, so just set to false
var memResult = await _search.MemorySearchAsync("183.192.62.65").ConfigureAwait(false);
var binarySearchResult = await _search.BinarySearchAsync("183.192.62.65").ConfigureAwait(false);
var bTreeSearchResult = await _search.BtreeSearchAsync("183.192.62.65").ConfigureAwait(false);
Assert.NotNull(memResult.Region);
Assert.NotNull(binarySearchResult.Region);
Assert.NotNull(bTreeSearchResult.Region);
Assert.Equal(memResult.Region, binarySearchResult.Region);
Assert.Equal(bTreeSearchResult.Region, memResult.Region);
}
public void Dispose()
{
_search.Dispose();
}
}
}

Binary file not shown.

View File

@ -0,0 +1,412 @@
//*******************************
// Created By Rocher Kong
// Github https://github.com/RocherKong
// Date 2018.02.09
//*******************************
using IP2Region.Models;
using System;
using System.IO;
using System.Text;
using System.Threading.Tasks;
namespace IP2Region
{
public class DbSearcher : IDisposable
{
const int BTREE_ALGORITHM = 1;
const int BINARY_ALGORITHM = 2;
const int MEMORY_ALGORITYM = 3;
private DbConfig _dbConfig = null;
/// <summary>
/// db file access handler
/// </summary>
private Stream _raf = null;
/// <summary>
/// header blocks buffer
/// </summary>
private long[] _headerSip = null;
private int[] _headerPtr = null;
private int _headerLength;
/// <summary>
/// super blocks info
/// </summary>
private long _firstIndexPtr = 0;
private long _lastIndexPtr = 0;
private int _totalIndexBlocks = 0;
/// <summary>
/// for memory mode
/// the original db binary string
/// </summary>
private byte[] _dbBinStr = null;
/// <summary>
/// Get by index ptr.
/// </summary>
private DataBlock GetByIndexPtr(long ptr)
{
_raf.Seek(ptr, SeekOrigin.Begin);
byte[] buffer = new byte[12];
_raf.Read(buffer, 0, buffer.Length);
long extra = Utils.GetIntLong(buffer, 8);
int dataLen = (int)((extra >> 24) & 0xFF);
int dataPtr = (int)((extra & 0x00FFFFFF));
_raf.Seek(dataPtr, SeekOrigin.Begin);
byte[] data = new byte[dataLen];
_raf.Read(data, 0, data.Length);
int city_id = (int)Utils.GetIntLong(data, 0);
string region = Encoding.UTF8.GetString(data, 4, data.Length - 4);
return new DataBlock(city_id, region, dataPtr);
}
public DbSearcher(DbConfig dbConfig, string dbFile)
{
if (_dbConfig == null)
{
_dbConfig = dbConfig;
}
_raf = new FileStream(dbFile, FileMode.Open, FileAccess.Read, FileShare.Read);
}
public DbSearcher(string dbFile) : this(null, dbFile) { }
public DbSearcher(DbConfig dbConfig, Stream dbFileStream)
{
if (_dbConfig == null)
{
_dbConfig = dbConfig;
}
_raf = dbFileStream;
}
public DbSearcher(Stream dbFileStream) : this(null, dbFileStream) { }
#region Sync Methods
/// <summary>
/// Get the region with a int ip address with memory binary search algorithm.
/// </summary>
private DataBlock MemorySearch(long ip)
{
int blen = IndexBlock.LENGTH;
if (_dbBinStr == null)
{
_dbBinStr = new byte[(int)_raf.Length];
_raf.Seek(0L, SeekOrigin.Begin);
_raf.Read(_dbBinStr, 0, _dbBinStr.Length);
//initialize the global vars
_firstIndexPtr = Utils.GetIntLong(_dbBinStr, 0);
_lastIndexPtr = Utils.GetIntLong(_dbBinStr, 4);
_totalIndexBlocks = (int)((_lastIndexPtr - _firstIndexPtr) / blen) + 1;
}
//search the index blocks to define the data
int l = 0, h = _totalIndexBlocks;
long sip = 0;
while (l <= h)
{
int m = (l + h) >> 1;
int p = (int)(_firstIndexPtr + m * blen);
sip = Utils.GetIntLong(_dbBinStr, p);
if (ip < sip)
{
h = m - 1;
}
else
{
sip = Utils.GetIntLong(_dbBinStr, p + 4);
if (ip > sip)
{
l = m + 1;
}
else
{
sip = Utils.GetIntLong(_dbBinStr, p + 8);
break;
}
}
}
//not matched
if (sip == 0) return null;
//get the data
int dataLen = (int)((sip >> 24) & 0xFF);
int dataPtr = (int)((sip & 0x00FFFFFF));
int city_id = (int)Utils.GetIntLong(_dbBinStr, dataPtr);
string region = Encoding.UTF8.GetString(_dbBinStr, dataPtr + 4, dataLen - 4);//new String(dbBinStr, dataPtr + 4, dataLen - 4, Encoding.UTF8);
return new DataBlock(city_id, region, dataPtr);
}
/// <summary>
/// Get the region throught the ip address with memory binary search algorithm.
/// </summary>
public DataBlock MemorySearch(string ip)
{
return MemorySearch(Utils.Ip2long(ip));
}
/// <summary>
/// Get the region with a int ip address with b-tree algorithm.
/// </summary>
private DataBlock BtreeSearch(long ip)
{
//check and load the header
if (_headerSip == null)
{
_raf.Seek(8L, SeekOrigin.Begin); //pass the super block
byte[] b = new byte[4096];
_raf.Read(b, 0, b.Length);
//fill the header
int len = b.Length >> 3, idx = 0; //b.lenght / 8
_headerSip = new long[len];
_headerPtr = new int[len];
long startIp, dataPtrTemp;
for (int i = 0; i < b.Length; i += 8)
{
startIp = Utils.GetIntLong(b, i);
dataPtrTemp = Utils.GetIntLong(b, i + 4);
if (dataPtrTemp == 0) break;
_headerSip[idx] = startIp;
_headerPtr[idx] = (int)dataPtrTemp;
idx++;
}
_headerLength = idx;
}
//1. define the index block with the binary search
if (ip == _headerSip[0])
{
return GetByIndexPtr(_headerPtr[0]);
}
else if (ip == _headerPtr[_headerLength - 1])
{
return GetByIndexPtr(_headerPtr[_headerLength - 1]);
}
int l = 0, h = _headerLength, sptr = 0, eptr = 0;
int m = 0;
while (l <= h)
{
m = (l + h) >> 1;
//perfectly matched, just return it
if (ip == _headerSip[m])
{
if (m > 0)
{
sptr = _headerPtr[m - 1];
eptr = _headerPtr[m];
}
else
{
sptr = _headerPtr[m];
eptr = _headerPtr[m + 1];
}
}
//less then the middle value
else if (ip < _headerSip[m])
{
if (m == 0)
{
sptr = _headerPtr[m];
eptr = _headerPtr[m + 1];
break;
}
else if (ip > _headerSip[m - 1])
{
sptr = _headerPtr[m - 1];
eptr = _headerPtr[m];
break;
}
h = m - 1;
}
else
{
if (m == _headerLength - 1)
{
sptr = _headerPtr[m - 1];
eptr = _headerPtr[m];
break;
}
else if (ip <= _headerSip[m + 1])
{
sptr = _headerPtr[m];
eptr = _headerPtr[m + 1];
break;
}
l = m + 1;
}
}
//match nothing just stop it
if (sptr == 0) return null;
//2. search the index blocks to define the data
int blockLen = eptr - sptr, blen = IndexBlock.LENGTH;
byte[] iBuffer = new byte[blockLen + blen]; //include the right border block
_raf.Seek(sptr, SeekOrigin.Begin);
_raf.Read(iBuffer, 0, iBuffer.Length);
l = 0; h = blockLen / blen;
long sip = 0;
int p = 0;
while (l <= h)
{
m = (l + h) >> 1;
p = m * blen;
sip = Utils.GetIntLong(iBuffer, p);
if (ip < sip)
{
h = m - 1;
}
else
{
sip = Utils.GetIntLong(iBuffer, p + 4);
if (ip > sip)
{
l = m + 1;
}
else
{
sip = Utils.GetIntLong(iBuffer, p + 8);
break;
}
}
}
//not matched
if (sip == 0) return null;
//3. get the data
int dataLen = (int)((sip >> 24) & 0xFF);
int dataPtr = (int)((sip & 0x00FFFFFF));
_raf.Seek(dataPtr, SeekOrigin.Begin);
byte[] data = new byte[dataLen];
_raf.Read(data, 0, data.Length);
int city_id = (int)Utils.GetIntLong(data, 0);
String region = Encoding.UTF8.GetString(data, 4, data.Length - 4);// new String(data, 4, data.Length - 4, "UTF-8");
return new DataBlock(city_id, region, dataPtr);
}
/// <summary>
/// Get the region throught the ip address with b-tree search algorithm.
/// </summary>
public DataBlock BtreeSearch(string ip)
{
return BtreeSearch(Utils.Ip2long(ip));
}
/// <summary>
/// Get the region with a int ip address with binary search algorithm.
/// </summary>
private DataBlock BinarySearch(long ip)
{
int blen = IndexBlock.LENGTH;
if (_totalIndexBlocks == 0)
{
_raf.Seek(0L, SeekOrigin.Begin);
byte[] superBytes = new byte[8];
_raf.Read(superBytes, 0, superBytes.Length);
//initialize the global vars
_firstIndexPtr = Utils.GetIntLong(superBytes, 0);
_lastIndexPtr = Utils.GetIntLong(superBytes, 4);
_totalIndexBlocks = (int)((_lastIndexPtr - _firstIndexPtr) / blen) + 1;
}
//search the index blocks to define the data
int l = 0, h = _totalIndexBlocks;
byte[] buffer = new byte[blen];
long sip = 0;
while (l <= h)
{
int m = (l + h) >> 1;
_raf.Seek(_firstIndexPtr + m * blen, SeekOrigin.Begin); //set the file pointer
_raf.Read(buffer, 0, buffer.Length);
sip = Utils.GetIntLong(buffer, 0);
if (ip < sip)
{
h = m - 1;
}
else
{
sip = Utils.GetIntLong(buffer, 4);
if (ip > sip)
{
l = m + 1;
}
else
{
sip = Utils.GetIntLong(buffer, 8);
break;
}
}
}
//not matched
if (sip == 0) return null;
//get the data
int dataLen = (int)((sip >> 24) & 0xFF);
int dataPtr = (int)((sip & 0x00FFFFFF));
_raf.Seek(dataPtr, SeekOrigin.Begin);
byte[] data = new byte[dataLen];
_raf.Read(data, 0, data.Length);
int city_id = (int)Utils.GetIntLong(data, 0);
String region = Encoding.UTF8.GetString(data, 4, data.Length - 4);//new String(data, 4, data.Length - 4, "UTF-8");
return new DataBlock(city_id, region, dataPtr);
}
/// <summary>
/// Get the region throught the ip address with binary search algorithm.
/// </summary>
public DataBlock BinarySearch(String ip)
{
return BinarySearch(Utils.Ip2long(ip));
}
#endregion
#region Async Methods
/// <summary>
/// Get the region throught the ip address with memory binary search algorithm.
/// </summary>
public Task<DataBlock> MemorySearchAsync(string ip)
{
return Task.FromResult(MemorySearch(ip));
}
/// <summary>
/// Get the region throught the ip address with b-tree search algorithm.
/// </summary>
public Task<DataBlock> BtreeSearchAsync(string ip)
{
return Task.FromResult(BtreeSearch(ip));
}
/// <summary>
/// Get the region throught the ip address with binary search algorithm.
/// </summary>
public Task<DataBlock> BinarySearchAsync(string ip)
{
return Task.FromResult(BinarySearch(ip));
}
#endregion
/// <summary>
/// Close the db.
/// </summary>
public void Close()
{
_headerSip = null;
_headerPtr = null;
_dbBinStr = null;
_raf.Close();
}
public void Dispose()
{
Close();
}
}
}

View File

@ -0,0 +1,26 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFrameworks>netstandard2.0;net46;net45;net47</TargetFrameworks>
<GeneratePackageOnBuild>true</GeneratePackageOnBuild>
<Authors>Rocher</Authors>
<Company>Rocher</Company>
<Description>准确率99.9%的ip地址定位库0.0x毫秒级查询, 支持Binary,B树,内存三种查询算法妈妈再也不用担心我的ip地址定位</Description>
<PackageReleaseNotes>ip2region - 最自由的ip地址查询库ip到地区的映射库提供Binary,B树和纯内存三种查询算法妈妈再也不用担心我的ip地址定位。</PackageReleaseNotes>
<Copyright>Rocher</Copyright>
<PackageTags>ip2region c# IP</PackageTags>
<RepositoryUrl>https://gitee.com/rocherkong/IP2Region</RepositoryUrl>
<Version>1.2.0</Version>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(TargetFramework)|$(Platform)'=='Debug|netstandard2.0|AnyCPU'">
<LangVersion>7</LangVersion>
</PropertyGroup>
<ItemGroup>
<None Update="DB\ip2region.db">
<CopyToOutputDirectory>Always</CopyToOutputDirectory>
</None>
</ItemGroup>
</Project>

View File

@ -0,0 +1,50 @@
//*******************************
// Created By Rocher Kong
// Github https://github.com/RocherKong
// Date 2018.02.09
//*******************************
namespace IP2Region.Models
{
public class DataBlock
{
#region Private Properties
public int CityID
{
get;
private set;
}
public string Region
{
get;
private set;
}
public int DataPtr
{
get;
private set;
}
#endregion
#region Constructor
public DataBlock(int city_id, string region, int dataPtr = 0)
{
CityID = city_id;
Region = region;
DataPtr = dataPtr;
}
public DataBlock(int city_id, string region):this(city_id,region,0)
{
}
#endregion
public override string ToString()
{
return $"{CityID}|{Region}|{DataPtr}";
}
}
}

View File

@ -0,0 +1,49 @@
//*******************************
// Created By Rocher Kong
// Github https://github.com/RocherKong
// Date 2018.02.09
//*******************************
using System;
namespace IP2Region.Models
{
public class DbMakerConfigException : Exception
{
public string ErrMsg { get; private set; }
public DbMakerConfigException(string errMsg)
{
ErrMsg = errMsg;
}
}
public class DbConfig
{
public int TotalHeaderSize
{
get;
private set;
}
public int indexBlockSize
{
get;
private set;
}
public DbConfig(int totalHeaderSize)
{
if ((totalHeaderSize % 8) != 0)
{
throw new DbMakerConfigException("totalHeaderSize must be times of 8");
}
TotalHeaderSize = totalHeaderSize;
//4 * 2048
indexBlockSize = 8192;
}
public DbConfig():this(8 * 2048)
{
}
}
}

View File

@ -0,0 +1,49 @@
//*******************************
// Created By Rocher Kong
// Github https://github.com/RocherKong
// Date 2018.02.09
//*******************************
namespace IP2Region.Models
{
internal class HeaderBlock
{
public long IndexStartIp
{
get;
private set;
}
public int IndexPtr
{
get;
private set;
}
public HeaderBlock(long indexStartIp, int indexPtr)
{
IndexStartIp = indexStartIp;
IndexPtr = indexPtr;
}
/// <summary>
/// Get the bytes for total storage
/// </summary>
/// <returns>
/// Bytes gotten.
/// </returns>
public byte[] GetBytes()
{
/*
* +------------+-----------+
* | 4bytes | 4bytes |
* +------------+-----------+
* start ip index ptr
*/
byte[] b = new byte[8];
Utils.WriteIntLong(b, 0, IndexStartIp);
Utils.WriteIntLong(b, 4, IndexPtr);
return b;
}
}
}

View File

@ -0,0 +1,65 @@
//*******************************
// Created By Rocher Kong
// Github https://github.com/RocherKong
// Date 2018.02.09
//*******************************
namespace IP2Region
{
internal class IndexBlock
{
public const int LENGTH = 12;
public long StartIP
{
get;
private set;
}
public long EndIp
{
get;
private set;
}
public uint DataPtr
{
get;
private set;
}
public int DataLen
{
get;
private set;
}
public IndexBlock(long startIp, long endIp, uint dataPtr, int dataLen)
{
StartIP = startIp;
EndIp = endIp;
DataPtr = dataPtr;
DataLen = dataLen;
}
public byte[] GetBytes()
{
/*
* +------------+-----------+-----------+
* | 4bytes | 4bytes | 4bytes |
* +------------+-----------+-----------+
* start ip end ip data ptr + len
*/
byte[] b = new byte[12];
Utils.WriteIntLong(b, 0, StartIP); //start ip
Utils.WriteIntLong(b, 4, EndIp); //end ip
//write the data ptr and the length
long mix = DataPtr | ((DataLen << 24) & 0xFF000000L);
Utils.WriteIntLong(b, 8, mix);
return b;
}
}
}

View File

@ -0,0 +1,122 @@
//*******************************
// Created By Rocher Kong
// Github https://github.com/RocherKong
// Date 2018.02.09
//*******************************
using System;
namespace IP2Region
{
public class IPInValidException : Exception
{
const string ERROR_MSG = "IP Illigel. Please input a valid IP.";
public IPInValidException() : base(ERROR_MSG) { }
}
internal static class Utils
{
/// <summary>
/// Write specfield bytes to a byte array start from offset.
/// </summary>
public static void Write(byte[] b, int offset, ulong v, int bytes)
{
for (int i = 0; i < bytes; i++)
{
b[offset++] = (byte)((v >> (8 * i)) & 0xFF);
}
}
/// <summary>
/// Write a int to a byte array.
/// </summary>
public static void WriteIntLong(byte[] b, int offset, long v)
{
b[offset++] = (byte)((v >> 0) & 0xFF);
b[offset++] = (byte)((v >> 8) & 0xFF);
b[offset++] = (byte)((v >> 16) & 0xFF);
b[offset] = (byte)((v >> 24) & 0xFF);
}
/// <summary>
/// Get a int from a byte array start from the specifiled offset.
/// </summary>
public static long GetIntLong(byte[] b, int offset)
{
return (
((b[offset++] & 0x000000FFL)) |
((b[offset++] << 8) & 0x0000FF00L) |
((b[offset++] << 16) & 0x00FF0000L) |
((b[offset] << 24) & 0xFF000000L)
);
}
/// <summary>
/// Get a int from a byte array start from the specifield offset.
/// </summary>
public static int GetInt3(byte[] b, int offset)
{
return (
(b[offset++] & 0x000000FF) |
(b[offset++] & 0x0000FF00) |
(b[offset] & 0x00FF0000)
);
}
public static int GetInt2(byte[] b, int offset)
{
return (
(b[offset++] & 0x000000FF) |
(b[offset] & 0x0000FF00)
);
}
public static int GetInt1(byte[] b, int offset)
{
return (
(b[offset] & 0x000000FF)
);
}
/// <summary>
/// String ip to long ip.
/// </summary>
public static long Ip2long(string ip)
{
string[] p = ip.Split('.');
if (p.Length != 4) throw new IPInValidException();
foreach (string pp in p)
{
if (pp.Length > 3) throw new IPInValidException();
if (!int.TryParse(pp, out int value) || value > 255)
{
throw new IPInValidException();
}
}
var bip1 = long.TryParse(p[0], out long ip1);
var bip2 = long.TryParse(p[1], out long ip2);
var bip3 = long.TryParse(p[2], out long ip3);
var bip4 = long.TryParse(p[3], out long ip4);
if (!bip1 || !bip2 || !bip3 || !bip4
|| ip4 > 255 || ip1 > 255 || ip2 > 255 || ip3 > 255
|| ip4 < 0 || ip1 < 0 || ip2 < 0 || ip3 < 0)
{
throw new IPInValidException();
}
long p1 = ((ip1 << 24) & 0xFF000000);
long p2 = ((ip2 << 16) & 0x00FF0000);
long p3 = ((ip3 << 8) & 0x0000FF00);
long p4 = ((ip4 << 0) & 0x000000FF);
return ((p1 | p2 | p3 | p4) & 0xFFFFFFFFL);
}
/// <summary>
/// Int to ip string.
/// </summary>
public static string Long2ip(long ip)
{
return $"{(ip >> 24) & 0xFF}.{(ip >> 16) & 0xFF}.{(ip >> 8) & 0xFF}.{ip & 0xFF}";
}
}
}

View File

@ -0,0 +1,6 @@
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<startup>
<supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5" />
</startup>
</configuration>

Binary file not shown.

View File

@ -0,0 +1,60 @@
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="15.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Import Project="$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props" Condition="Exists('$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props')" />
<PropertyGroup>
<Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
<Platform Condition=" '$(Platform)' == '' ">AnyCPU</Platform>
<ProjectGuid>{2D73D7EB-AE36-4039-A839-DA40209052A4}</ProjectGuid>
<OutputType>Exe</OutputType>
<RootNamespace>IP2Region_NetFx_Test</RootNamespace>
<AssemblyName>IP2Region_NetFx_Test</AssemblyName>
<TargetFrameworkVersion>v4.5</TargetFrameworkVersion>
<FileAlignment>512</FileAlignment>
</PropertyGroup>
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
<PlatformTarget>AnyCPU</PlatformTarget>
<DebugSymbols>true</DebugSymbols>
<DebugType>full</DebugType>
<Optimize>false</Optimize>
<OutputPath>bin\Debug\</OutputPath>
<DefineConstants>DEBUG;TRACE</DefineConstants>
<ErrorReport>prompt</ErrorReport>
<WarningLevel>4</WarningLevel>
</PropertyGroup>
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
<PlatformTarget>AnyCPU</PlatformTarget>
<DebugType>pdbonly</DebugType>
<Optimize>true</Optimize>
<OutputPath>bin\Release\</OutputPath>
<DefineConstants>TRACE</DefineConstants>
<ErrorReport>prompt</ErrorReport>
<WarningLevel>4</WarningLevel>
</PropertyGroup>
<ItemGroup>
<Reference Include="System" />
<Reference Include="System.Core" />
<Reference Include="System.Xml.Linq" />
<Reference Include="System.Data.DataSetExtensions" />
<Reference Include="Microsoft.CSharp" />
<Reference Include="System.Data" />
<Reference Include="System.Net.Http" />
<Reference Include="System.Xml" />
</ItemGroup>
<ItemGroup>
<Compile Include="Program.cs" />
<Compile Include="Properties\AssemblyInfo.cs" />
</ItemGroup>
<ItemGroup>
<None Include="App.config" />
<None Include="DB\ip2region.db">
<CopyToOutputDirectory>Always</CopyToOutputDirectory>
</None>
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\IP2Region\IP2Region.csproj">
<Project>{a27d09f4-4da8-4925-90ec-43e5f5288630}</Project>
<Name>IP2Region</Name>
</ProjectReference>
</ItemGroup>
<Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
</Project>

View File

@ -0,0 +1,17 @@
using IP2Region;
using System;
namespace IP2Region_NetFx_Test
{
class Program
{
static void Main(string[] args)
{
using (var _search = new DbSearcher(Environment.CurrentDirectory + @"\DB\ip2region.db"))
{
Console.WriteLine(_search.MemorySearch("183.192.62.65").Region);
Console.Read();
}
}
}
}

View File

@ -0,0 +1,35 @@
using System.Reflection;
using System.Runtime.InteropServices;
// 有关程序集的一般信息由以下
// 控制。更改这些特性值可修改
// 与程序集关联的信息。
[assembly: AssemblyTitle("IP2Region_NetFx_Test")]
[assembly: AssemblyDescription("")]
[assembly: AssemblyConfiguration("")]
[assembly: AssemblyCompany("")]
[assembly: AssemblyProduct("IP2Region_NetFx_Test")]
[assembly: AssemblyCopyright("Copyright © 2018")]
[assembly: AssemblyTrademark("")]
[assembly: AssemblyCulture("")]
// 将 ComVisible 设置为 false 会使此程序集中的类型
//对 COM 组件不可见。如果需要从 COM 访问此程序集中的类型
//请将此类型的 ComVisible 特性设置为 true。
[assembly: ComVisible(false)]
// 如果此项目向 COM 公开,则下列 GUID 用于类型库的 ID
[assembly: Guid("2d73d7eb-ae36-4039-a839-da40209052a4")]
// 程序集的版本信息由下列四个值组成:
//
// 主版本
// 次版本
// 生成号
// 修订号
//
// 可以指定所有值,也可以使用以下所示的 "*" 预置版本号和修订号
// 方法是按如下所示使用“*”: :
// [assembly: AssemblyVersion("1.0.*")]
[assembly: AssemblyVersion("1.0.0.0")]
[assembly: AssemblyFileVersion("1.0.0.0")]

42
binding/c#/README.md Normal file
View File

@ -0,0 +1,42 @@
# IP2Region C# Client
## How To Use
### 1.Install from Nuget. Support .Net Framework (>=4.5) And netstandard 2.0(.net core)
```powershell
Install-Package IP2Region
```
### 2.Init DbSearcher with [the newest DBFile](https://github.com/lionsoul2014/ip2region/blob/master/data/ip2region.db) Downloaded into your project, and invoke your search methods as below:
```csharp
using (var _search = new DbSearcher(Environment.CurrentDirectory + @"\DB\ip2region.db"))
{
_search.MemorySearch("183.192.62.65").Region;
_search.BinarySearch("183.192.62.65").Region;
_search.BtreeSearch("183.192.62.65").Region;
}
```
For async methods (Methods with the suffix "Async"), just put them with `async` and `await`.
If you don't need the `SynchronizedContext`, please use `ConfigureAwait(false)`.
For more about this, please read https://msdn.microsoft.com/en-us/magazine/jj991977.aspx.
```csharp
public async Task SearchAsync_Test()
{
// We don't need the synchronizeContext, so just set to false.
// So as for BinarySearchAsync and BtreeSearchAsync
var memResult = await _search.MemorySearchAsync("183.192.62.65").ConfigureAwait(false);
}
```
## Test Result(From /IP2Region.Test.Benchmark)
Method | Mean | Error | StdDev | Rank |
------------------- |---------: |----------: |----------: |-----: |
MemorySearch | 10.66 us | 0.1424 us | 0.1332 us | 1 |
MemorySearch_Async | 10.90 us | 0.2060 us | 0.2023 us | 2 |
BinarySearch | 52.22 us | 0.6403 us | 0.5347 us | 5 |
BinarySearch_Async | 53.03 us | 1.0271 us | 0.9608 us | 6 |
BtreeSearch | 19.05 us | 0.2464 us | 0.2305 us | 3 |
BtreeSearch_Async | 19.40 us | 0.3820 us | 0.6690 us | 4 |
Because the speed of the program lookup has been fast, the asynchronous method here is just the packaging of the synchronization method, the essence is the synchronization method, the asynchronous method is not recommended, and asynchronous will bring the performance loss, unless you need the asynchronous call format.

26
binding/c/Makefile Normal file
View File

@ -0,0 +1,26 @@
A = @
CFLAGS= -g
CC= gcc
LIBS= -I./
LDFLAGS=
RM = rm -f
MV = mv
testSearcher: ip2region.o
$(A) $(CC) $(CFLAGS) $(LDFLAGS) $(LIBS) -o testSearcher testSearcher.c ip2region.c
$(A) $(RM) ip2region.o testSearcher.o
db:
$(A) cd ../../;\
export LANG=en_US.UTF-8;\
java -jar dbMaker-1.2.2.jar -src ./data/ip.merge.txt -region ./data/global_region.csv -dst .;\
$(MV) ./ip2region.db ./binding/c/;\
cd ./binding/c
all: db testSearcher
clean:
$(A) $(RM) testSearcher
.PHONY: all clean testSearcher db

View File

@ -15,7 +15,7 @@
* *
* @param dbFile path * @param dbFile path
*/ */
IP2R_API uint_t ip2region_create(ip2region_t ip2rObj, char *dbFile) IP2R_API uint_t ip2region_create(ip2region_t ip2rObj, const char *dbFile)
{ {
memset(ip2rObj, 0x00, sizeof(ip2region_entry)); memset(ip2rObj, 0x00, sizeof(ip2region_entry));
ip2rObj->headerLen = 0; ip2rObj->headerLen = 0;
@ -55,10 +55,15 @@ IP2R_API uint_t ip2region_create(ip2region_t ip2rObj, char *dbFile)
*/ */
IP2R_API uint_t ip2region_destroy(ip2region_t ip2rObj) IP2R_API uint_t ip2region_destroy(ip2region_t ip2rObj)
{ {
if ( ip2rObj->HeaderSip != NULL ) {
IP2R_FREE(ip2rObj->HeaderSip); IP2R_FREE(ip2rObj->HeaderSip);
ip2rObj->HeaderSip = NULL; ip2rObj->HeaderSip = NULL;
}
if ( ip2rObj->HeaderPtr != NULL ) {
IP2R_FREE(ip2rObj->HeaderPtr); IP2R_FREE(ip2rObj->HeaderPtr);
ip2rObj->HeaderPtr = NULL; ip2rObj->HeaderPtr = NULL;
}
// close the db file resource // close the db file resource
if ( ip2rObj->dbHandler != NULL ) { if ( ip2rObj->dbHandler != NULL ) {
@ -149,7 +154,7 @@ IP2R_API uint_t ip2region_memory_search(ip2region_t ip2rObj, uint_t ip, databloc
return 1; return 1;
} }
IP2R_API uint_t ip2region_memory_search_string(ip2region_t ip2rObj, char *ip, datablock_t datablock) IP2R_API uint_t ip2region_memory_search_string(ip2region_t ip2rObj, const char *ip, datablock_t datablock)
{ {
return ip2region_memory_search(ip2rObj, ip2long(ip), datablock); return ip2region_memory_search(ip2rObj, ip2long(ip), datablock);
} }
@ -226,7 +231,7 @@ IP2R_API uint_t ip2region_binary_search(ip2region_t ip2rObj, uint_t ip, databloc
return 1; return 1;
} }
IP2R_API uint_t ip2region_binary_search_string(ip2region_t ip2rObj, char *ip, datablock_t datablock) IP2R_API uint_t ip2region_binary_search_string(ip2region_t ip2rObj, const char *ip, datablock_t datablock)
{ {
return ip2region_binary_search(ip2rObj, ip2long(ip), datablock); return ip2region_binary_search(ip2rObj, ip2long(ip), datablock);
} }
@ -355,7 +360,7 @@ IP2R_API uint_t ip2region_btree_search(ip2region_t ip2rObj, uint_t ip, datablock
return 1; return 1;
} }
IP2R_API uint_t ip2region_btree_search_string(ip2region_t ip2rObj, char *ip, datablock_t datablock) IP2R_API uint_t ip2region_btree_search_string(ip2region_t ip2rObj, const char *ip, datablock_t datablock)
{ {
return ip2region_btree_search(ip2rObj, ip2long(ip), datablock); return ip2region_btree_search(ip2rObj, ip2long(ip), datablock);
} }
@ -367,7 +372,7 @@ IP2R_API uint_t ip2region_btree_search_string(ip2region_t ip2rObj, char *ip, dat
* @param offset * @param offset
* @return uint_t * @return uint_t
*/ */
IP2R_API uint_t getUnsignedInt(char *buffer, int offset) IP2R_API uint_t getUnsignedInt(const char *buffer, int offset)
{ {
return ( return (
((buffer[offset ]) & 0x000000FF) | ((buffer[offset ]) & 0x000000FF) |
@ -383,13 +388,19 @@ IP2R_API uint_t getUnsignedInt(char *buffer, int offset)
* @param ip * @param ip
* @return uint_t * @return uint_t
*/ */
IP2R_API uint_t ip2long(char *ip) IP2R_API uint_t ip2long(const char *ip)
{ {
int i = 0, p = 24; int i = 0, p = 24;
char buffer[4], *cs = ip; char buffer[4];
const char *cs = ip;
uint_t ipval = 0; uint_t ipval = 0;
while ( *cs != '\0' ) { while ( *cs != '\0' ) {
if ( *cs == ' ' ) {
cs++;
continue;
}
if ( *cs == '.' ) { if ( *cs == '.' ) {
//single part length limit //single part length limit
if ( i > 3 ) { if ( i > 3 ) {

View File

@ -43,7 +43,9 @@ typedef unsigned int uint_t;
typedef unsigned long ulong_t; typedef unsigned long ulong_t;
#define INDEX_BLOCK_LENGTH 12 #define INDEX_BLOCK_LENGTH 12
#define TOTAL_HEADER_LENGTH 4096
//thats 8 * 1024
#define TOTAL_HEADER_LENGTH 8192
/* /*
* ip2region properties struct * ip2region properties struct
@ -78,7 +80,7 @@ typedef datablock_entry * datablock_t;
* @param ip2rObj * @param ip2rObj
* @param dbFile path * @param dbFile path
*/ */
IP2R_API uint_t ip2region_create(ip2region_t, char *); IP2R_API uint_t ip2region_create(ip2region_t, const char *);
/** /**
* destroy the specified ip2region object * destroy the specified ip2region object
@ -96,7 +98,7 @@ IP2R_API uint_t ip2region_destroy(ip2region_t);
* @date 2016/06/30 * @date 2016/06/30
*/ */
IP2R_API uint_t ip2region_memory_search(ip2region_t, uint_t, datablock_t); IP2R_API uint_t ip2region_memory_search(ip2region_t, uint_t, datablock_t);
IP2R_API uint_t ip2region_memory_search_string(ip2region_t, char *, datablock_t); IP2R_API uint_t ip2region_memory_search_string(ip2region_t, const char *, datablock_t);
/** /**
@ -108,7 +110,7 @@ IP2R_API uint_t ip2region_memory_search_string(ip2region_t, char *, datablock_t)
* @return uint_t * @return uint_t
*/ */
IP2R_API uint_t ip2region_binary_search(ip2region_t, uint_t, datablock_t); IP2R_API uint_t ip2region_binary_search(ip2region_t, uint_t, datablock_t);
IP2R_API uint_t ip2region_binary_search_string(ip2region_t, char *, datablock_t); IP2R_API uint_t ip2region_binary_search_string(ip2region_t, const char *, datablock_t);
/** /**
* get the region associated with the specified ip address with b-tree algorithm * get the region associated with the specified ip address with b-tree algorithm
@ -119,7 +121,7 @@ IP2R_API uint_t ip2region_binary_search_string(ip2region_t, char *, datablock_t)
* @return uint_t * @return uint_t
*/ */
IP2R_API uint_t ip2region_btree_search(ip2region_t, uint_t, datablock_t); IP2R_API uint_t ip2region_btree_search(ip2region_t, uint_t, datablock_t);
IP2R_API uint_t ip2region_btree_search_string(ip2region_t, char *, datablock_t); IP2R_API uint_t ip2region_btree_search_string(ip2region_t, const char *, datablock_t);
/** /**
* get a unsinged long(4bytes) from a specified buffer start from the specified offset * get a unsinged long(4bytes) from a specified buffer start from the specified offset
@ -128,7 +130,7 @@ IP2R_API uint_t ip2region_btree_search_string(ip2region_t, char *, datablock_t);
* @param offset * @param offset
* @return uint_t * @return uint_t
*/ */
IP2R_API uint_t getUnsignedInt(char *, int); IP2R_API uint_t getUnsignedInt(const char *, int);
/** /**
* string ip to long * string ip to long
@ -136,7 +138,7 @@ IP2R_API uint_t getUnsignedInt(char *, int);
* @param ip * @param ip
* @return uint_t * @return uint_t
*/ */
IP2R_API uint_t ip2long(char *); IP2R_API uint_t ip2long(const char *);
/** /**
* long to string ip * long to string ip

View File

@ -49,7 +49,7 @@ int main( int argc, char **argv )
datablock_entry datablock; datablock_entry datablock;
char *dbFile = NULL, *algorithm = NULL; char *dbFile = NULL, *algorithm = NULL;
char line[256]; char line[256];
uint_t (*func_ptr)(ip2region_t, char *, datablock_t); uint_t (*func_ptr)(ip2region_t, const char *, datablock_t);
double s_time, c_time; double s_time, c_time;
memset(&datablock, 0x00, sizeof(datablock_entry)); memset(&datablock, 0x00, sizeof(datablock_entry));

328
binding/c_mmap/ip2region.c Normal file
View File

@ -0,0 +1,328 @@
/**
* ip2region implementation with mmap support created by Leo Ma
*
* @see #ip2region.h
* @author chenxin<chenxin619315@gmail.com>
* @author Leo Ma<http://git.oschina.net/begeekmyfriend>
* @date 2017/01/12
*/
#include "ip2region.h"
#include <unistd.h>
#include <fcntl.h>
#include <sys/stat.h>
#include <sys/mman.h>
/**
* create a new ip2region object
*
* @param dbFile path
*/
IP2R_API uint_t ip2region_create(ip2region_t ip2rObj, char *dbFile)
{
struct stat st;
memset(ip2rObj, 0x00, sizeof(ip2region_entry));
ip2rObj->dbFile = IP2R_MALLOC(1024);
strcpy(ip2rObj->dbFile, dbFile);
int fd = open(dbFile, O_RDONLY);
if ( fd < 0 ) {
//fprintf(stderr, "Fail to open the db file %s\n", ip2rObj->dbFile);
return 0;
}
if ( fstat(fd, &st) < 0 ) {
//fprintf(stderr, "Fail to stat the db file %s\n", ip2rObj->dbFile);
return 0;
}
ip2rObj->dbSize = st.st_size;
ip2rObj->dbBinStr = mmap(NULL, ip2rObj->dbSize, PROT_READ, MAP_PRIVATE, fd, 0);
if ( ip2rObj->dbBinStr == (void *) -1 ) {
//fprintf(stderr, "Fail to map the db file %s\n", ip2rObj->dbFile);
return 0;
}
// Close db file as soon as possible.
close(fd);
ip2rObj->headerLen = 0;
ip2rObj->HeaderSip = (uint_t *) IP2R_MALLOC(TOTAL_HEADER_LENGTH);
if ( ip2rObj->HeaderSip == NULL ) {
return 0;
}
ip2rObj->HeaderPtr = (uint_t *) IP2R_MALLOC(TOTAL_HEADER_LENGTH);
if ( ip2rObj->HeaderPtr == NULL ) {
IP2R_FREE(ip2rObj->HeaderSip);
return 0;
}
ip2rObj->firstIndexPtr = getUnsignedInt(ip2rObj->dbBinStr, 0);
ip2rObj->lastIndexPtr = getUnsignedInt(ip2rObj->dbBinStr, 4);
ip2rObj->totalBlocks = (ip2rObj->lastIndexPtr-ip2rObj->firstIndexPtr)/INDEX_BLOCK_LENGTH + 1;
return 1;
}
/**
* destroy the specifield ip2region object
*
* @param ip2region_t
*/
IP2R_API uint_t ip2region_destroy(ip2region_t ip2rObj)
{
IP2R_FREE(ip2rObj->dbFile);
IP2R_FREE(ip2rObj->HeaderSip);
IP2R_FREE(ip2rObj->HeaderPtr);
//free the db binary string
if ( ip2rObj->dbBinStr != NULL ) {
munmap(ip2rObj->dbBinStr, ip2rObj->dbSize);
}
memset(ip2rObj, 0x00, sizeof(ip2region_entry));
return 1;
}
/**
* get the region associated with the specified ip address with the memory binary search algorithm
*
* @param ip2rObj
* @param ip
* @param datablock
*/
IP2R_API uint_t ip2region_binary_search(ip2region_t ip2rObj, uint_t ip, datablock_t datablock)
{
int l, h, m, p;
uint_t sip, eip, dptr;
int dataLen, dataptr;
char *buffer;
l = 0; h = ip2rObj->totalBlocks; dptr = 0;
while ( l <= h ) {
m = (l + h) >> 1;
p = ip2rObj->firstIndexPtr + m * INDEX_BLOCK_LENGTH;
buffer = ip2rObj->dbBinStr + p;
sip = getUnsignedInt(buffer, 0);
if ( ip < sip ) {
h = m - 1;
} else {
eip = getUnsignedInt(buffer, 4);
if ( ip > eip ) {
l = m + 1;
} else {
dptr = getUnsignedInt(buffer, 8);
break;
}
}
}
if ( dptr == 0 ) return 0;
//get the data
dataLen = ((dptr >> 24) & 0xFF);
dataptr = (dptr & 0x00FFFFFF);
buffer = ip2rObj->dbBinStr + dataptr;
//fill the data to the datablock
datablock->city_id = getUnsignedInt(buffer, 0);
dataLen -= 4; //reduce the length of the city_id
memcpy(datablock->region, buffer + 4, dataLen);
datablock->region[dataLen] = '\0';
return 1;
}
IP2R_API uint_t ip2region_binary_search_string(ip2region_t ip2rObj, char *ip, datablock_t datablock)
{
return ip2region_binary_search(ip2rObj, ip2long(ip), datablock);
}
/**
* get the region associated with the specifield ip address with b-tree algorithm
*
* @param ip2rObj
* @param ip
* @param datablock
* @return uint_t
*/
IP2R_API uint_t ip2region_btree_search(ip2region_t ip2rObj, uint_t ip, datablock_t datablock)
{
int i, j;
int l, m, h, p, sptr, eptr, indexBlockLen, dataLen, dataptr;
uint_t sip, eip, idxptr, dptr;
char *buffer;
if ( ip2rObj->headerLen == 0 ) {
buffer = ip2rObj->dbBinStr + 8;
for ( i = j = 0; i < TOTAL_HEADER_LENGTH; i += 8, j++ ) {
sip = getUnsignedInt(buffer, i);
idxptr = getUnsignedInt(buffer, i + 4);
if ( idxptr == 0 ) break;
ip2rObj->HeaderSip[j] = sip;
ip2rObj->HeaderPtr[j] = idxptr;
}
ip2rObj->headerLen = j;
}
//search the header block to define the index block
l = 0; h = ip2rObj->headerLen; sptr = 0; eptr = 0;
while ( l <= h ) {
m = ((l + h) >> 1);
//perfetc matched, just return it
if ( ip == ip2rObj->HeaderSip[m] ) {
if ( m > 0 ) {
sptr = ip2rObj->HeaderPtr[m-1];
eptr = ip2rObj->HeaderPtr[m ];
} else {
sptr = ip2rObj->HeaderPtr[m ];
eptr = ip2rObj->HeaderPtr[m+1];
}
break;
}
//less then the middle value
if ( ip < ip2rObj->HeaderSip[m] ) {
if ( m == 0 ) {
sptr = ip2rObj->HeaderPtr[m ];
eptr = ip2rObj->HeaderPtr[m+1];
break;
} else if ( ip > ip2rObj->HeaderSip[m-1] ) {
sptr = ip2rObj->HeaderPtr[m-1];
eptr = ip2rObj->HeaderPtr[m ];
break;
}
h = m - 1;
} else {
if ( m == ip2rObj->headerLen - 1 ) {
sptr = ip2rObj->HeaderPtr[m-1];
eptr = ip2rObj->HeaderPtr[m ];
break;
} else if ( ip <= ip2rObj->HeaderSip[m+1] ) {
sptr = ip2rObj->HeaderPtr[m ];
eptr = ip2rObj->HeaderPtr[m+1];
break;
}
l = m + 1;
}
}
//not matched just stop it
if ( sptr == 0 ) return 0;
indexBlockLen = eptr - sptr;
buffer = ip2rObj->dbBinStr + sptr;
dptr = 0; l = 0; h = indexBlockLen / INDEX_BLOCK_LENGTH;
while ( l <= h ) {
m = ((l + h) >> 1);
p = m * INDEX_BLOCK_LENGTH;
sip = getUnsignedInt(buffer, p);
if ( ip < sip ) {
h = m - 1;
} else {
eip = getUnsignedInt(buffer, p + 4);
if ( ip > eip ) {
l = m + 1;
} else {
dptr = getUnsignedInt(buffer, p + 8);
break;
}
}
}
if ( dptr == 0 ) return 0;
dataLen = ((dptr >> 24) & 0xFF);
dataptr = (dptr & 0x00FFFFFF);
buffer = ip2rObj->dbBinStr + dataptr;
datablock->city_id = getUnsignedInt(buffer, 0);
dataLen -= 4;
memcpy(datablock->region, buffer + 4, dataLen);
datablock->region[dataLen] = '\0';
return 1;
}
IP2R_API uint_t ip2region_btree_search_string(ip2region_t ip2rObj, char *ip, datablock_t datablock)
{
return ip2region_btree_search(ip2rObj, ip2long(ip), datablock);
}
/**
* get a unsinged long(4bytes) from a specifield buffer start from the specifield offset
*
* @param buffer
* @param offset
* @return uint_t
*/
IP2R_API uint_t getUnsignedInt(char *buffer, int offset)
{
return (
((buffer[offset ]) & 0x000000FF) |
((buffer[offset+1] << 8) & 0x0000FF00) |
((buffer[offset+2] << 16) & 0x00FF0000) |
((buffer[offset+3] << 24) & 0xFF000000)
);
}
/**
* string ip to long
*
* @param ip
* @return uint_t
*/
IP2R_API uint_t ip2long(char *ip)
{
int i = 0, p = 24;
char buffer[4], *cs = ip;
uint_t ipval = 0;
while ( *cs != '\0' ) {
if ( *cs == '.' ) {
//single part length limit
if ( i > 3 ) {
ipval = 0;
break;
}
if ( p < 0 ) break;
buffer[i] = '\0';
ipval |= (atoi(buffer) << p);
p -= 8;
i = 0;
} else {
buffer[i++] = *cs;
}
cs++;
}
//append the rest parts
if ( i > 3 ) return 0;
buffer[i] = '\0';
ipval |= atoi(buffer);
return ipval;
}
/**
* long to string ip
*
* @param ip
* @param buffer
* @return uint_t(1 for success and 0 for failed)
*/
IP2R_API uint_t long2ip(uint_t ip, char *buffer)
{
return 0;
}

138
binding/c_mmap/ip2region.h Normal file
View File

@ -0,0 +1,138 @@
/**
* ip2region header file
*
* @author chenxin<chenxin619315@gmail.com>
* @author Leo Ma<http://git.oschina.net/begeekmyfriend>
*/
#ifndef _IP2REGION_H
#define _IP2REGION_H
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
//yat, just take it as this way, 99 percent and they works find
#if ( defined(_WIN32) || defined(_WINDOWS_) || defined(__WINDOWS_) )
# define OS_WINDOW
#else
# define OS_LINUX
#endif
#ifdef OS_WINDOW
# define IP2R_API extern __declspec(dllexport)
#else
/*platform shared library statement :: unix*/
# define IP2R_API extern
#endif
#define print(str) printf("%s", str)
#define println(str) printf("%s\n", str)
/*
* memory allocation macro definition.
* cause we should use emalloc,ecalloc .ege. in php.
* so you could make it better apdat the php environment.
*/
#define IP2R_CALLOC(_bytes, _blocks) calloc(_bytes, _blocks)
#define IP2R_MALLOC(_bytes) malloc(_bytes)
#define IP2R_FREE(_ptr) free(_ptr)
typedef unsigned short ushort_t;
typedef unsigned char uchar_t;
typedef unsigned int uint_t;
typedef unsigned long ulong_t;
#define INDEX_BLOCK_LENGTH 12
#define TOTAL_HEADER_LENGTH 8192
/*
* ip2region properties struct
*/
typedef struct {
uint_t *HeaderSip; //header start ip blocks
uint_t *HeaderPtr; //header ptr blocks
uint_t headerLen; //header block number
char *dbFile; //path of db file
char *dbBinStr; //db binary string for memory search mode
uint_t dbSize;
uint_t firstIndexPtr; //first index ptr
uint_t lastIndexPtr; //last index ptr
uint_t totalBlocks; //total index blocks number
} ip2region_entry;
typedef ip2region_entry * ip2region_t;
/*
* data block
*/
typedef struct {
uint_t city_id;
char region[256];
} datablock_entry;
typedef datablock_entry * datablock_t;
/**
* create a new ip2region object
*
* @param ip2rObj
* @param dbFile path
*/
IP2R_API uint_t ip2region_create(ip2region_t, char *);
/**
* destroy the specified ip2region object
*
* @param ip2region_t
*/
IP2R_API uint_t ip2region_destroy(ip2region_t);
/**
* get the region associated with the specified ip address with memory binary search algorithm
*
* @param ip2region_t
* @param uint_t
* @param datablock_t
* @date 2016/06/30
*/
IP2R_API uint_t ip2region_binary_search(ip2region_t, uint_t, datablock_t);
IP2R_API uint_t ip2region_binary_search_string(ip2region_t, char *, datablock_t);
/**
* get the region associated with the specified ip address with b-tree algorithm
*
* @param ip2rObj
* @param ip
* @param datablock
* @return uint_t
*/
IP2R_API uint_t ip2region_btree_search(ip2region_t, uint_t, datablock_t);
IP2R_API uint_t ip2region_btree_search_string(ip2region_t, char *, datablock_t);
/**
* get a unsinged long(4bytes) from a specified buffer start from the specified offset
*
* @param buffer
* @param offset
* @return uint_t
*/
IP2R_API uint_t getUnsignedInt(char *, int);
/**
* string ip to long
*
* @param ip
* @return uint_t
*/
IP2R_API uint_t ip2long(char *);
/**
* long to string ip
*
* @param ip
* @param buffer
* @return uint_t(1 for success and 0 for failed)
*/
IP2R_API uint_t long2ip(uint_t, char *);
#endif /*end ifndef*/

View File

@ -0,0 +1,99 @@
/**
* test ip2region searcher program
*
* @author chenxin<chenxin619315@gmail.com>
* @date 2015-10-30
*/
#include "ip2region.h"
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <sys/time.h>
#define __PRINT_ABOUT__ \
println("+-------------------------------------+"); \
println("| ip2region test program |"); \
println("| Author: chenxin619315@gmail.com. |"); \
println("| Type 'quit' to exit the program. |"); \
println("+-------------------------------------+");
//read a line from a command line.
static char *getLine( FILE *fp, char *__dst )
{
register int c;
register char *cs;
cs = __dst;
while ( ( c = getc( fp ) ) != EOF ) {
if ( c == '\n' ) break;
*cs++ = c;
}
*cs = '\0';
return ( c == EOF && cs == __dst ) ? NULL : __dst;
}
static double getTime()
{
struct timeval tv;
struct timezone tz;
gettimeofday(&tv, &tz);
return (tv.tv_sec * 1000 + ((double)tv.tv_usec)/1000);
}
int main( int argc, char **argv )
{
ip2region_entry ip2rEntry;
datablock_entry datablock;
char *dbFile = NULL, *algorithm = NULL;
char line[256];
uint_t (*func_ptr)(ip2region_t, char *, datablock_t);
double s_time, c_time;
memset(&datablock, 0x00, sizeof(datablock_entry));
if ( argc < 2 ) {
printf("Usage: a.out [ip2region db file path] [algorithm]\n");
return 0;
}
dbFile = argv[1];
algorithm = "B-tree";
func_ptr = ip2region_btree_search_string;
if ( argc >= 3 ) {
if ( strcmp(argv[2], "binary") == 0 ) {
algorithm = "File";
func_ptr = ip2region_binary_search_string;
}
}
//create a new ip2rObj
printf("+--initializing %s ... \n", algorithm);
if ( ip2region_create(&ip2rEntry, dbFile) == 0 ) {
println("Error: Fail to create the ip2region object\n");
return 0;
}
__PRINT_ABOUT__;
while ( 1 ) {
print("ip2region>> ");
getLine( stdin, line );
if ( strlen(line) < 2 ) continue;
if ( strcasecmp( line, "quit" ) == 0 ) {
println("+--Bye!");
break;
}
s_time = getTime();
func_ptr(&ip2rEntry, line, &datablock);
c_time = getTime() - s_time;
printf("%d|%s in %.5f millseconds\n", datablock.city_id, datablock.region, c_time);
}
//destory the ip2rObj
ip2region_destroy(&ip2rEntry);
return 0;
}

View File

@ -3,7 +3,7 @@
#### 获取 #### 获取
``` ```
go get github.com/mohong122/ip2region/binding/golang go get github.com/lionsoul2014/ip2region/binding/golang
``` ```
@ -17,7 +17,7 @@ package main
import ( import (
"fmt" "fmt"
"github.com/mohong122/ip2region/binding/golang" "github.com/lionsoul2014/ip2region/binding/golang/ip2region"
) )
func main() { func main() {
@ -64,7 +64,7 @@ BenchmarkBinarySearch-4| 30000 | 42680 ns/op
``` ```
cd /binging/golang cd /binging/golang
go run main.go ../../data/ip2Region.db go run main.go ../../data/ip2region.db
Or Or

View File

@ -10,7 +10,7 @@ import (
const ( const (
INDEX_BLOCK_LENGTH = 12 INDEX_BLOCK_LENGTH = 12
TOTAL_HEADER_LENGTH = 4096 TOTAL_HEADER_LENGTH = 8192
) )
var err error var err error

View File

@ -2,7 +2,7 @@ package main
import ( import (
"os" "os"
"github.com/mohong122/ip2region/binding/golang" "github.com/lionsoul2014/ip2region/binding/golang/ip2region"
"bufio" "bufio"
"fmt" "fmt"
"strings" "strings"

View File

@ -9,7 +9,7 @@
<property name="jars" value="${basedir}"/> <property name="jars" value="${basedir}"/>
<property name="sources" value="${basedir}/src"/> <property name="sources" value="${basedir}/src"/>
<property name="classes" value="${basedir}/classes"/> <property name="classes" value="${basedir}/classes"/>
<property name="version" value="1.2"/> <property name="version" value="1.4"/>
<property name="api" value="${basedir}/doc"/> <property name="api" value="${basedir}/doc"/>
<mkdir dir="${classes}"/> <mkdir dir="${classes}"/>
<mkdir dir="${api}"/> <mkdir dir="${api}"/>

190
binding/java/pom.xml Normal file
View File

@ -0,0 +1,190 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.lionsoul</groupId>
<artifactId>ip2region</artifactId>
<version>1.7.2</version>
<packaging>jar</packaging>
<name>ip2region</name>
<url>http://gitee.com/lionsoul/ip2region/</url>
<description>Open source internel address locator</description>
<licenses>
<license>
<name>The Apache Software License, Version 2.0</name>
<url>http://www.apache.org/licenses/LICENSE-2.0.txt</url>
<distribution>repo</distribution>
</license>
</licenses>
<scm>
<url>git@gitee.com:lionsoul/ip2region.git</url>
<connection>scm:git:git@gitee.com:lionsoul/ip2region.git</connection>
<developerConnection>scm:git:git@gitee.com:lionsoul/ip2region.git</developerConnection>
</scm>
<developers>
<developer>
<id>lionsoul</id>
<name>chenxin</name>
<email>chenxin619315@gmail.com</email>
</developer>
</developers>
<issueManagement>
<url>http://gitee.com/lionsoul/ip2region/issues</url>
<system>gitee issues</system>
</issueManagement>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
</properties>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-source-plugin</artifactId>
<version>2.1.2</version>
<executions>
<execution>
<id>attach-sources</id>
<phase>package</phase>
<goals>
<goal>jar</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<version>2.9</version>
<executions>
<execution>
<id>attach-javadocs</id>
<phase>package</phase>
<goals>
<goal>jar</goal>
</goals>
<configuration>
<additionalparam>${javadoc.opts}</additionalparam>
</configuration>
</execution>
</executions>
<configuration>
<failOnError>false</failOnError>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>1.4</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>org.lionsoul.ip2region.test.TestSearcher</mainClass>
</transformer>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
<profiles>
<profile>
<id>java8-doclint-disabled</id>
<activation>
<jdk>[1.8,)</jdk>
</activation>
<properties>
<javadoc.opts>-Xdoclint:none</javadoc.opts>
</properties>
</profile>
<profile>
<id>release</id>
<build>
<plugins>
<!-- Source -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-source-plugin</artifactId>
<version>2.2.1</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>jar-no-fork</goal>
</goals>
</execution>
</executions>
</plugin>
<!-- Javadoc -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<version>2.9.1</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>jar</goal>
</goals>
<configuration>
<additionalparam>${javadoc.opts}</additionalparam>
</configuration>
</execution>
</executions>
</plugin>
<!-- GPG -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-gpg-plugin</artifactId>
<version>1.5</version>
<executions>
<execution>
<phase>verify</phase>
<goals>
<goal>sign</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
<distributionManagement>
<snapshotRepository>
<id>oss-parent</id>
<url>https://oss.sonatype.org/content/repositories/snapshots/</url>
</snapshotRepository>
<repository>
<id>oss-parent</id>
<url>https://oss.sonatype.org/service/local/staging/deploy/maven2/</url>
</repository>
</distributionManagement>
</profile>
</profiles>
</project>

View File

@ -32,12 +32,12 @@ public class DbConfig
} }
this.totalHeaderSize = totalHeaderSize; this.totalHeaderSize = totalHeaderSize;
this.indexBlockSize = 4096; //4 * 1024 this.indexBlockSize = 8192; //4 * 2048
} }
public DbConfig() throws DbMakerConfigException public DbConfig() throws DbMakerConfigException
{ {
this(8192); this(8 * 2048);
} }
public int getTotalHeaderSize() public int getTotalHeaderSize()

View File

@ -58,6 +58,23 @@ public class DbSearcher
raf = new RandomAccessFile(dbFile, "r"); raf = new RandomAccessFile(dbFile, "r");
} }
/**
* construct method with self-define std ip2region bianry string support
* Thanks to the issue from Wendal at https://gitee.com/lionsoul/ip2region/issues/IILFL
*
* @param dbConfig
* @param dbBinStr
*/
public DbSearcher(DbConfig dbConfig, byte[] dbBinStr)
{
this.dbConfig = dbConfig;
this.dbBinStr = dbBinStr;
firstIndexPtr = Util.getIntLong(dbBinStr, 0);
lastIndexPtr = Util.getIntLong(dbBinStr, 4);
totalIndexBlocks = (int)((lastIndexPtr - firstIndexPtr)/IndexBlock.getIndexBlockLength()) + 1;
}
/** /**
* get the region with a int ip address with memory binary search algorithm * get the region with a int ip address with memory binary search algorithm
* *
@ -163,8 +180,8 @@ public class DbSearcher
//check and load the header //check and load the header
if ( HeaderSip == null ) { if ( HeaderSip == null ) {
raf.seek(8L); //pass the super block raf.seek(8L); //pass the super block
//byte[] b = new byte[dbConfig.getTotalHeaderSize()]; byte[] b = new byte[dbConfig.getTotalHeaderSize()];
byte[] b = new byte[4096]; // byte[] b = new byte[4096];
raf.readFully(b, 0, b.length); raf.readFully(b, 0, b.length);
//fill the header //fill the header

443
binding/lua/Ip2region.lua Normal file
View File

@ -0,0 +1,443 @@
--[[
Ip2region lua binding
@author chenxin<chenxin619315@gmail.com>
@date 2018/10/02
]]--
require("bit32");
local INDEX_BLOCK_LENGTH = 12;
local TOTAL_HEADER_LENGTH = 8192;
local _M = {
dbFile = "",
dbFileHandler = "",
dbBinStr = "",
HeaderSip = "",
HeaderPtr = "",
headerLen = 0,
firstIndexPtr = 0,
lastIndexPtr = 0,
totalBlocks = 0
};
_G["Ip2region"] = _M;
-- set the __index to itself
_M.__index = _M;
-- set the print meta-method
_M.__tostring = function(table)
local t = {
"dbFile=" .. table.dbFile,
-- "dbFileHandler=" .. table.dbFileHandler,
"headerLen=" .. table.headerLen,
"firstIndexPtr" .. table.firstIndexPtr,
"lastIndexPtr" .. table.lastIndexPtr,
"totalBlocks" .. table.totalBlocks
};
return table.concat(t, ",");
end
--[[
construct method
@param obj
@return Ip2region object
--]]
function _M.new(dbFile)
obj = {};
setmetatable(obj, _M);
obj.dbFile = dbFile;
return obj;
end
--[[
internal function to get a integer from a binary string
@param dbBinStr
@param idx
@return Integer
]]--
function getLong(bs, idx)
local a1 = string.byte(string.sub(bs, idx, idx));
local a2 = bit32.lshift(string.byte(string.sub(bs, idx+1, idx+1)), 8);
local a3 = bit32.lshift(string.byte(string.sub(bs, idx+2, idx+2)), 16);
local a4 = bit32.lshift(string.byte(string.sub(bs, idx+3, idx+3)), 24);
local val = bit32.bor(a1, a2);
val = bit32.bor(val, a3);
val = bit32.bor(val, a4);
return val;
end
--[[
internal function to convert the string ip to a long value
@param ip
@return Integer
]]--
function _M.ip2long(self, ip)
-- dynamic arguments checking
-- to support object.ip2long and object:ip2long access
if ( type(self) == "string") then
ip = self;
end
local ini = 1;
local iip = 0;
local off = 24;
while true do
local pos = string.find(ip, '.', ini, true);
if ( not pos ) then
break;
end
local sub = string.sub(ip, ini, pos - 1);
if ( string.len(sub) < 1 ) then
return nil;
end
iip = bit32.bor(iip, bit32.lshift(tonumber(sub), off));
ini = pos + 1;
off = off - 8;
end
-- check if it is a valid ip address
if ( off ~= 0 or ini > string.len(ip) ) then
return nil;
end
local sub = string.sub(ip, ini);
if ( string.len(sub) < 1 ) then
return nil;
end
return bit32.bor(iip, bit32.lshift(tonumber(sub), off));
end
--[[
internal function to get the whole content of a file
@param file
@return String
]]--
function get_file_contents(file)
local fi = io.input(file);
if ( not fi ) then
return nil;
end
local str = io.read("*a");
io.close();
return str;
end
--[[
set the current db file path
@param dbFile
]]--
function _M:setDbFile(dbFile)
self.dbFile = dbFile;
end
--[[
all the db binary string will be loaded into memory
then search the memory only and this will a lot faster than disk base search
@Note: invoke it once before put it to public invoke could make it thread safe
@param ip
@return table or nil for failed
]]--
function _M:memorySearch(ip)
-- string ip conversion
if ( type(ip) == "string" ) then
ip = self:ip2long(ip);
if ( ip == nil ) then
return nil;
end
end;
-- check and load the binary string for the first time
if ( self.dbBinStr == "" ) then
self.dbBinStr = get_file_contents(self.dbFile);
if ( not self.dbBinStr ) then
return nil;
end
self.firstIndexPtr = getLong(self.dbBinStr, 1);
self.lastIndexPtr = getLong(self.dbBinStr, 5);
self.totalBlocks = (self.lastIndexPtr - self.firstIndexPtr)/INDEX_BLOCK_LENGTH + 1;
end
-- binary search to define the data
local l = 0;
local h = self.totalBlocks;
local dataPtr = 0;
while ( l <= h ) do
local m = math.floor((l + h) / 2);
local p = self.firstIndexPtr + m * INDEX_BLOCK_LENGTH;
local sip = getLong(self.dbBinStr, p + 1);
if ( ip < sip ) then
h = m - 1;
else
local eip = getLong(self.dbBinStr, p + 5); -- 4 + 1
if ( ip > eip ) then
l = m + 1;
else
dataPtr = getLong(self.dbBinStr, p + 9); -- 8 + 1
break;
end
end
end
-- not matched just stop it here
if ( dataPtr == 0 ) then return nil end
-- get the data
local dataLen = bit32.band(bit32.rshift(dataPtr, 24), 0xFF);
dataPtr = bit32.band(dataPtr, 0x00FFFFFF);
local dptr = dataPtr + 5; -- 4 + 1
return {
city_id = getLong(self.dbBinStr, dataPtr),
region = string.sub(self.dbBinStr, dptr, dptr + dataLen - 5)
};
end
--[[
get the data block through the specified ip address
or long ip numeric with binary search algorithm
@param ip
@return table or nil for failed
]]--
function _M:binarySearch(ip)
-- check and conver the ip address
if ( type(ip) == "string" ) then
ip = self:ip2long(ip);
if ( ip == nil ) then
return nil;
end
end
if ( self.totalBlocks == 0 ) then
-- check and open the original db file
self.dbFileHandler = io.open(self.dbFile, "r");
if ( not self.dbFileHandler ) then
return nil;
end
self.dbFileHandler:seek("set", 0);
local superBlock = self.dbFileHandler:read(8);
self.firstIndexPtr = getLong(superBlock, 1); -- 0 + 1
self.lastIndexPtr = getLong(superBlock, 5); -- 4 + 1
self.totalBlocks = (self.lastIndexPtr-self.firstIndexPtr)/INDEX_BLOCK_LENGTH + 1;
end
-- binary search to define the data
local l = 0;
local h = self.totalBlocks;
local dataPtr = 0;
while ( l <= h ) do
local m = math.floor((l + h) / 2);
local p = m * INDEX_BLOCK_LENGTH;
self.dbFileHandler:seek("set", self.firstIndexPtr + p);
local buffer = self.dbFileHandler:read(INDEX_BLOCK_LENGTH);
local sip = getLong(buffer, 1); -- 0 + 1
if ( ip < sip ) then
h = m - 1;
else
local eip = getLong(buffer, 5); -- 4 + 1
if ( ip > eip ) then
l = m + 1;
else
dataPtr = getLong(buffer, 9); -- 8 + 1
break;
end
end
end
-- not matched just stop it here
if ( dataPtr == 0 ) then return nil; end
-- get the data
local dataLen = bit32.band(bit32.rshift(dataPtr, 24), 0xFF);
dataPtr = bit32.band(dataPtr, 0x00FFFFFF);
self.dbFileHandler:seek("set", dataPtr);
local data = self.dbFileHandler:read(dataLen);
return {
city_id = getLong(data, 1), -- 0 + 1
region = string.sub(data, 5) -- 4 + 1
};
end
--[[
get the data block associated with the specified ip with b-tree search algorithm
@param ip
@return table or nil for failed
]]--
function _M:btreeSearch(ip)
-- string ip to integer conversion
if ( type(ip) == "string" ) then
ip = self:ip2long(ip);
if ( ip == nil ) then
return nil;
end
end
-- check and load the header
if ( self.headerLen == 0 ) then
-- check and open the original db file
self.dbFileHandler = io.open(self.dbFile, 'r');
if ( not self.dbFileHandler ) then
return nil;
end
self.dbFileHandler:seek("set", 8);
local buffer = self.dbFileHandler:read(TOTAL_HEADER_LENGTH);
-- fill the header
local i = 0;
local idx = 0;
self.HeaderSip = {};
self.HeaderPtr = {};
for i=0, TOTAL_HEADER_LENGTH, 8 do
local startIp = getLong(buffer, i + 1); -- 0 + 1
local dataPtr = getLong(buffer, i + 5); -- 4 + 1
if ( dataPtr == 0 ) then
break;
end
table.insert(self.HeaderSip, startIp);
table.insert(self.HeaderPtr, dataPtr);
idx = idx + 1;
end
self.headerLen = idx;
end
-- 1. define the index block with the binary search
local l = 0;
local h = self.headerLen;
local sptr = 0;
local eptr = 0;
while ( l <= h ) do
local m = math.floor((l + h) / 2);
-- perfetc matched, just return it
if ( ip == self.HeaderSip[m] ) then
if ( m > 0 ) then
sptr = self.HeaderPtr[m-1];
eptr = self.HeaderPtr[m ];
else
sptr = self.HeaderPtr[m ];
eptr = self.HeaderPtr[m+1];
end
break;
end
-- less then the middle value
if ( ip < self.HeaderSip[m] ) then
if ( m == 0 ) then
sptr = self.HeaderPtr[m ];
eptr = self.HeaderPtr[m+1];
break;
elseif ( ip > self.HeaderSip[m-1] ) then
sptr = self.HeaderPtr[m-1];
eptr = self.HeaderPtr[m ];
break;
end
h = m - 1;
else
if ( m == self.headerLen - 1 ) then
sptr = self.HeaderPtr[m-1];
eptr = self.HeaderPtr[m ];
break;
elseif ( ip <= self.HeaderSip[m+1] ) then
sptr = self.HeaderPtr[m ];
eptr = self.HeaderPtr[m+1];
break;
end
l = m + 1;
end
end
-- match nothing just stop it
if ( sptr == 0 ) then return nil; end
-- 2. search the index blocks to define the data
self.dbFileHandler:seek("set", sptr);
local blockLen = eptr - sptr;
local index = self.dbFileHandler:read(blockLen + INDEX_BLOCK_LENGTH);
local dataPtr = 0;
l = 0;
h = blockLen / INDEX_BLOCK_LENGTH;
while ( l <= h ) do
local m = math.floor((l + h) / 2);
local p = m * INDEX_BLOCK_LENGTH;
local sip = getLong(index, p + 1); -- 0 + 1
if ( ip < sip ) then
h = m - 1;
else
local eip = getLong(index, p + 5); -- 4 + 1
if ( ip > eip ) then
l = m + 1;
else
dataPtr = getLong(index, p + 9); -- 8 + 1
break;
end
end
end
-- not matched
if ( dataPtr == 0 ) then return nil; end
-- 3. get the data
local dataLen = bit32.band(bit32.rshift(dataPtr, 24), 0xFF);
dataPtr = bit32.band(dataPtr, 0x00FFFFFF);
self.dbFileHandler:seek("set", dataPtr);
local data = self.dbFileHandler:read(dataLen);
return {
city_id = getLong(data, 1), -- 0 + 1
region = string.sub(data, 5) -- 4 + 1
};
end
--[[
close the object and do the basic gc
]]--
function _M.close(self)
if ( self.dbFileHandler ~= "" ) then
self.dbFileHandler:close();
end
if ( self.dbBinStr ~= "" ) then
self.dbBinStr = nil;
end
end
return _M;

59
binding/lua/README.md Normal file
View File

@ -0,0 +1,59 @@
# Lua ip2region binding
### 一,如何测试
* 1, cd到ip2region/binding/lua/根目录
* 2, 运行testSearcher测试程序
```shell
lua testSearcher.lua ../../data/ip2region.db
```
* 3, 输入ip地址开始测试即可
```shell
initializing btree
+----------------------------------+
| ip2region test script |
| Author: chenxin619315@gmail.com |
| Type 'quit' to exit program |
+----------------------------------+
ip2region>> 1.2.3.4
0|美国|0|华盛顿|0|0 in 8.001000 millseconds
ip2region>> 101.233.153.103
2163|中国|0|广东省|深圳市|鹏博士 in 0.176000 millseconds
ip2region>>
```
### 二,如何使用
* 1, 将Ip2region.lua拷贝到你的package.path下
* 2, 通过如下流程在你的lua程序中使用
```lua
-- 包含模块
local Ip2region = require "Ip2region";
-- 创建查询对象
-- 设置ip2region.db的文件地址dbFile表示ip2region.db数据库文件的地址
-- 注意new方法是通过“.”调用,而不是“:”
local ip2region = Ip2region.new("ip2region.db file path");
-- 也可以通过如下方式设置dbFile
-- ip2region.dbFile = "ip2region.db file path";
-- ip2region.setDbFile("ip2region.db file path");
local data;
-- 查询,备注,尽量请使用“:”调用方法,使用“.”需要主动传递ip2region对象参数
-- 1binary查询
data = ip2region:binarySearch("101.233.153.103");
-- 2btree查询
data = ip2region:btreeSearch("101.233.153.103");
-- 3memory查询
data = ip2region:memorySearch("101.233.153.103");
-- 返回结果如下
print("city_id=", data.city_id, "region=", data.region);
```
### 三,备注
* 1Ip2region模块的位运算依赖了bit32模块这个是lua 5.2才开始支持的。

View File

@ -0,0 +1,85 @@
--[[
ip2region lua client test script
@author chenxin<chenxin619315@gmail.com>
]]--
-- check the command line arguments
if ( not arg[1] ) then
print([[
Usage: lua testSearcher.lua [ip2region db file] [algorithm]
+-Optional Algorithm: binary, b-tree, memory]]);
os.exit();
end
local Ip2region = require "Ip2region";
-- local cjson = require "cjson";
-- local socket = require "socket";
-- check and parse the dbFile and the method algorithm
-- Create a new ip2region object by the new interface
local ip2region = Ip2region.new(arg[1]);
-- reset the dbFile by the follow two ways:
-- ip2region.dbFile = arg[1];
-- ip2region:setDbFile(arg[1]);
local algorithm = "btree";
if ( arg[2] ~= nil ) then
local arg_2 = string.lower(arg[2]);
if ( arg_2 ~= "binary" and arg_2 ~= "memory" ) then
algorithm = "binary";
elseif ( arg_2 == "memory" ) then
algorithm = "memory";
end
end
-- local data = searcher:memorySearch("120.79.17.142");
-- local data = searcher:binarySearch("120.79.17.142");
-- local data = searcher:btreeSearch("120.79.17.142");
print("initializing " .. algorithm ..[[
+----------------------------------+
| ip2region test script |
| Author: chenxin619315@gmail.com |
| Type 'quit' to exit program |
+----------------------------------+]]
);
while ( true ) do
io.write("ip2region>> ");
io.input(io.stdin);
local line = io.read();
if ( line == nil ) then
-- do nothing
break;
elseif ( line == "quit" ) then
break;
elseif ( ip2region.ip2long(line) == nil ) then
print("Invalid ip address=", line);
else
local data;
local s_time = os.clock();
if ( algorithm == "btree" ) then
data = ip2region:btreeSearch(line);
elseif ( algorithm == "binary" ) then
data = ip2region:binarySearch(line);
elseif ( algorithm == "memory" ) then
data = ip2region:memorySearch(line);
end
local cost_time = (os.clock() - s_time) * 1000; -- to millseconds
if ( data == nil ) then
io.write("Failed for ip=", line, " is it a valid ip address ?");
else
io.write(string.format("%u|%s in %5f millseconds\n", data.city_id, data.region, cost_time));
end
end
end
-- close the object
-- print(ip2region);
ip2region:close();

28
binding/lua_c/Makefile Normal file
View File

@ -0,0 +1,28 @@
# ip2region luc c module Make file
# @author chenxin<chenxin619315@gmail.com>
# @date 2018/10/05
# @Note
# Please modify the LIBS and the LIB_DIR to fit you system
#
VER = 5.2
CC = gcc
LIBS = -I ../c/ -I /usr/include/lua$(VER)/
FFLAGS = -O2 -Wall -fPIC
SO_FILE = Ip2region.so
LIB_DIR = /usr/local/share/lua/$(VER)
all: ../c/ip2region.c ../c/ip2region.h lua_ip2region.c
$(CC) $(FFLAGS) $(LIBS) ../c/ip2region.c lua_ip2region.c -fPIC -shared -o $(SO_FILE)
install:
sudo mkdir -p $(LIB_DIR); \
sudo cp $(SO_FILE) $(LIB_DIR);\
echo "install Ip2region to $(LIB_DIR) successfully.";\
clean:
find . -name \*.so | xargs rm -f
find . -name \*.o | xargs rm -f
.PHONY: clean

74
binding/lua_c/README.md Normal file
View File

@ -0,0 +1,74 @@
# Lua ip2region c module binding
### 一,如何安装
* 1, cd到ip2region/binding/lua_c/根目录
* 2, 运行如下命令
```
make
sudo make install
```
* 3, 关于Makefile你可能需要更改里面的VER(版本号)和LIB_DIR(lua的so lib目录)来适应你的系统
```shell
VER = 5.2
CC = gcc
LIBS = -I ../c/ -I /usr/include/lua$(VER)/
FFLAGS = -O2 -Wall -fPIC
SO_FILE = Ip2region.so
LIB_DIR = /usr/local/share/lua/$(VER)
```
### 二,如何测试
* 1, cd到ip2region/binding/lua_c/根目录
* 2, 运行testSearcher测试程序
```shell
lua testSearcher.lua ../../data/ip2region.db
```
* 3, 输入ip地址开始测试即可
```shell
initializing btree
+----------------------------------+
| ip2region test script |
| Author: chenxin619315@gmail.com |
| Type 'quit' to exit program |
+----------------------------------+
ip2region>> 1.2.3.4
0|美国|0|华盛顿|0|0 in 0.100000 millseconds
ip2region>> 101.233.153.103
2163|中国|0|广东省|深圳市|鹏博士 in 0.045000 millseconds
ip2region>>
```
### 三,如何使用
* 1, 先参考第一步安装把Ip2region.so安装到你系统lua的lib目录下
* 2, 通过如下流程在你的lua程序中使用
```lua
-- 包含模块
local Ip2region = require "Ip2region";
-- 创建查询对象
-- 设置ip2region.db的文件地址dbFile表示ip2region.db数据库文件的地址
-- 注意new方法是通过“.”调用,而不是“:”
local ip2region = Ip2region.new("ip2region.db file path");
local data;
-- 查询,备注,尽量请使用“:”调用方法,使用“.”需要主动传递ip2region对象参数
-- 1binary查询
data = ip2region:binarySearch("101.233.153.103");
-- 2btree查询
data = ip2region:btreeSearch("101.233.153.103");
-- 3memory查询
data = ip2region:memorySearch("101.233.153.103");
-- 返回结果如下
print("city_id=", data.city_id, "region=", data.region);
```
### 四,备注
* 1c模块的Ip2region以来binding/c/的实现请保持binding/c/存在并且和lua_c模块同目录。
* 2Ip2region的c模块拥有和c模块几乎等同的性能生产建议使用lua_c模块代替纯lua模块的使用。

View File

@ -0,0 +1,232 @@
/**
* Ip2region lua c binding
*
* @author chenxin<chenxin619315@gmail.com>
* @date 2018/10/04
*/
#include <stdio.h>
#include <stdlib.h>
#include <lua.h>
#include <lauxlib.h>
#include "../c/ip2region.h"
#define L_METATABLE_NAME "Ip2region_MT"
/** create a new ip2region object with a specified dbFile */
static int lua_ip2region_new(lua_State *L)
{
ip2region_entry *self;
const char *dbFile;
/* Check the arguments are valid */
dbFile = luaL_checkstring(L, 1);
if ( dbFile == NULL ) {
luaL_error(L, "dbFile cannot be empty");
}
/* Create the user data and push it onto the stack */
self = (ip2region_entry *) lua_newuserdata(L, sizeof(ip2region_entry));
/* Push the metatable onto the stack */
luaL_getmetatable(L, L_METATABLE_NAME);
/* Set the metatable on the userdata */
lua_setmetatable(L, -2);
/* Initialize the entry */
ip2region_create(self, dbFile);
return 1;
}
/** destroy the specified ip2region instance */
static int lua_ip2region_destroy(lua_State *L)
{
ip2region_entry *self;
self = (ip2region_entry *) luaL_checkudata(L, 1, L_METATABLE_NAME);
ip2region_destroy(self);
return 0;
}
#define get_search_args(L, entry, ip) \
do { \
luaL_argcheck(L, lua_gettop(L) == 2, 2, "Object and ip address needed"); \
entry = (ip2region_entry *) luaL_checkudata(L, 1, L_METATABLE_NAME); \
ip = luaL_checkstring(L, 2); \
} while (0); \
#define set_search_result(L, data) \
do { \
lua_newtable(L); \
lua_pushinteger(L, data.city_id); \
lua_setfield(L, -2, "city_id"); \
lua_pushfstring(L, "%s", data.region); \
lua_setfield(L, -2, "region"); \
} while (0); \
/** ip2region_memory_search wrapper */
static int lua_ip2region_memory_search(lua_State *L)
{
ip2region_entry *self;
const char *addr;
datablock_entry data;
// luaL_argcheck(L, lua_gettop(L) == 2, 2, "object and ip address needed");
// self = (ip2region_entry *) luaL_checkudata(L, 1, L_METATABLE_NAME);
// addr = luaL_checkstring(L, 2);
/* Check and get the search parameters */
get_search_args(L, self, addr);
/* do the memory search */
if ( ip2region_memory_search(self, ip2long(addr), &data) == 0 ) {
lua_pushnil(L);
return 1;
}
// lua_newtable(L);
// lua_pushinteger(L, data.city_id);
// lua_setfield(L, -2, "city_id");
// lua_pushfstring(L, "%s", data.region);
// lua_setfield(L, -2, "region");
set_search_result(L, data);
return 1;
}
/** ip2region_binary_search wrapper */
static int lua_ip2region_binary_search(lua_State *L)
{
ip2region_entry *self;
const char *addr;
datablock_entry data;
/* Check and get the search parameters */
get_search_args(L, self, addr);
/* Do the binary search */
if ( ip2region_binary_search(self, ip2long(addr), &data) == 0 ) {
lua_pushnil(L);
return 1;
}
/* Set the return value */
set_search_result(L, data);
return 1;
}
/** ip2region_btree_search wrapper */
static int lua_ip2region_btree_search(lua_State *L)
{
ip2region_entry *self;
const char *addr;
datablock_entry data;
/* Check and get the search parameters */
get_search_args(L, self, addr);
/* Do the btree search */
if ( ip2region_btree_search(self, ip2long(addr), &data) == 0 ) {
lua_pushnil(L);
return 1;
}
/* Set the return value */
set_search_result(L, data);
return 1;
}
/** ip2long wrapper */
static int lua_ip2long(lua_State *L)
{
int argc;
const char *addr;
uint_t ipval;
argc = lua_gettop(L);
if ( argc == 1 ) {
addr = luaL_checkstring(L, 1);
} else {
luaL_checkudata(L, 1, L_METATABLE_NAME);
addr = luaL_checkstring(L, 2);
}
if ( (ipval = ip2long(addr)) == 0 ) {
lua_pushnil(L);
return 1;
}
lua_pushinteger(L, (ipval & 0x7FFFFFFF));
return 1;
}
static int lua_ip2region_tostring(lua_State *L)
{
ip2region_entry *self;
self = (ip2region_entry *) luaL_checkudata(L, 1, L_METATABLE_NAME);
/* Push the string to return to lua */
lua_pushfstring(L,
"dbFile=%s, headerLen=%d, fristIndexPtr=%d, lastIndexPtr=%d, totalBlocks=%d",
self->dbFile, self->headerLen, self->firstIndexPtr,
self->lastIndexPtr, self->totalBlocks
);
return 1;
}
/** module method array */
static const struct luaL_Reg ip2region_methods[] = {
// { "new", lua_ip2region_new },
// { "ip2long", lua_ip2long },
{ "memorySearch", lua_ip2region_memory_search },
{ "binarySearch", lua_ip2region_binary_search },
{ "btreeSearch", lua_ip2region_btree_search },
{ "close", lua_ip2region_destroy },
{ "__gc", lua_ip2region_destroy },
{ "__tostring", lua_ip2region_tostring },
{ NULL, NULL },
};
/** module function array */
static const struct luaL_Reg ip2region_functions[] = {
{ "new", lua_ip2region_new },
{ "ip2long", lua_ip2long },
{ NULL, NULL }
};
/** module open function interface */
int luaopen_Ip2region(lua_State *L)
{
/* Create a metatable and push it onto the stack */
luaL_newmetatable(L, L_METATABLE_NAME);
/* Duplicate the metatable on the stack */
lua_pushvalue(L, -1);
/* Pop the first metatable off the stack
* and assign it to the __index of the second one.
* so we set the metatable to the table itself.
*/
lua_setfield(L, -2, "__index");
/* Set the methods fo the metatable that could and should be
* access via object:func in lua block
*/
luaL_setfuncs(L, ip2region_methods, 0);
luaL_setfuncs(L, ip2region_functions, 0);
/* Finally register the object.func functions
* into the table witch at the top of the stack */
// luaL_newlib(L, ip2region_functions);
return 1;
}

View File

@ -0,0 +1,86 @@
--[[
ip2region lua client test script
@author chenxin<chenxin619315@gmail.com>
]]--
-- check the command line arguments
if ( not arg[1] ) then
print([[
Usage: lua testSearcher.lua [ip2region db file] [algorithm]
+-Optional Algorithm: binary, b-tree, memory]]);
os.exit();
end
local Ip2region = require "Ip2region";
-- local cjson = require "cjson";
-- local socket = require "socket";
-- check and parse the dbFile and the method algorithm
-- Create a new ip2region object by the new interface
local ip2region = Ip2region.new(arg[1]);
-- reset the dbFile by the follow two ways:
-- ip2region.dbFile = arg[1];
-- ip2region:setDbFile(arg[1]);
local algorithm = "btree";
if ( arg[2] ~= nil ) then
local arg_2 = string.lower(arg[2]);
if ( arg_2 ~= "binary" and arg_2 ~= "memory" ) then
algorithm = "binary";
elseif ( arg_2 == "memory" ) then
algorithm = "memory";
end
end
-- local data = searcher:memorySearch("120.79.17.142");
-- local data = searcher:binarySearch("120.79.17.142");
-- local data = searcher:btreeSearch("120.79.17.142");
print("initializing " .. algorithm ..[[
+----------------------------------+
| ip2region test script |
| Author: chenxin619315@gmail.com |
| Type 'quit' to exit program |
+----------------------------------+]]
);
while ( true ) do
io.write("ip2region>> ");
io.input(io.stdin);
local line = io.read();
if ( line == nil ) then
-- do nothing
break;
elseif ( line == "quit" ) then
break;
elseif ( Ip2region.ip2long(line) == nil ) then
print("Invalid ip address=", line);
else
local data;
local s_time = os.clock();
if ( algorithm == "btree" ) then
data = ip2region:btreeSearch(line);
elseif ( algorithm == "binary" ) then
data = ip2region:binarySearch(line);
elseif ( algorithm == "memory" ) then
data = ip2region:memorySearch(line);
end
local cost_time = (os.clock() - s_time) * 1000; -- to millseconds
if ( data == nil ) then
io.write("Failed for ip=", line, " is it a valid ip address ?");
else
io.write(string.format("%u|%s in %5f millseconds\n", data.city_id, data.region, cost_time));
end
end
end
-- close the object
-- Also the lua gc will invoke it automatically
-- print(ip2region);
ip2region:close();

2
binding/nodejs/.gitignore vendored Normal file
View File

@ -0,0 +1,2 @@
node_modules/
data/

View File

@ -0,0 +1,4 @@
{
"singleQuote": true,
"tabWidth": 4
}

View File

@ -1,3 +1,86 @@
# nodejs 客户端 # nodejs 客户端
现已实现同步查询,异步查询未实现,用法参考 test.js ## Install
```
npm install lionsoul-ip2region --save
```
## Example
```
const searcher = require('lionsoul-ip2region').create();
searcher.btreeSearchSync('xxx.xxx.xxx.xxx')
// => { city: 2163, region: '中国|0|广东省|深圳市|联通' }
```
## 实现情况:
现已实现同步和异步查询,具体使用方法可以参考 `nodejs\tests\constructorTest.spec.js` 和`nodejs\tests\createTest.spec.js`。
## 如何贡献?
你可以任意修改代码,但必须确保通过全部的单元测试。要保证通过全部的单元测试,请在 Nodejs 控制台下切换到 nodejs 目录:
1在此之前请先运行 `npm i` 确保你已经安装了各类初始化第三方工具。
2然后运行 `npm run coverage` 确保你的代码可以通过全部测试(必要时可以添加测试)。
```bash
D:\Projects\ip2region\binding\nodejs>npm run coverage
> ip2region@0.0.1 coverage D:\Projects\ip2region\binding\nodejs
> npm run test && jest --coverage
> ip2region@0.0.1 test D:\Projects\ip2region\binding\nodejs
> jest
PASS tests\constructorTest.spec.js
PASS tests\createTest.spec.js
PASS tests\exceptionTest.spec.js
Snapshot Summary
168 snapshots written in 2 test suites.
Test Suites: 3 passed, 3 total
Tests: 14 passed, 14 total
Snapshots: 168 added, 168 total
Time: 1.645s
Ran all test suites.
PASS tests\constructorTest.spec.js
PASS tests\createTest.spec.js
PASS tests\exceptionTest.spec.js
----------------------|----------|----------|----------|----------|-------------------|
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s |
----------------------|----------|----------|----------|----------|-------------------|
All files | 92.34 | 80.77 | 96 | 93.83 | |
nodejs | 91.95 | 80.26 | 95.65 | 93.51 | |
ip2region.js | 91.95 | 80.26 | 95.65 | 93.51 |... 09,410,460,484 |
nodejs/tests/utils | 100 | 100 | 100 | 100 | |
asyncFor.js | 100 | 100 | 100 | 100 | |
fetchMainVersion.js | 100 | 100 | 100 | 100 | |
testData.js | 100 | 100 | 100 | 100 | |
----------------------|----------|----------|----------|----------|-------------------|
Test Suites: 3 passed, 3 total
Tests: 14 passed, 14 total
Snapshots: 168 passed, 168 total
Time: 1.792s
Ran all test suites.
```
3使用benchmark测试结果如下
```bash
D:\Projects\ip2region\binding\nodejs>node D:\Projects\ip2region\binding\nodejs\tests\benchmarkTests\main.js
MemorySearchSync x 55,969 ops/sec ±2.22% (90 runs sampled)
BinarySearchSync x 610 ops/sec ±5.41% (77 runs sampled)
BtreeSearchSync x 2,439 ops/sec ±6.93% (69 runs sampled)
MemorySearch x 2,924 ops/sec ±0.67% (85 runs sampled)
BinarySearch x 154 ops/sec ±2.20% (69 runs sampled)
BtreeSearch x 294 ops/sec ±2.58% (76 runs sampled)
Rand Name Time (in milliseconds)
1 MemorySearchSync 0.018
2 MemorySearch 0.342
3 BtreeSearchSync 0.410
4 BinarySearchSync 1.639
5 BtreeSearch 3.407
6 BinarySearch 6.497
```

11
binding/nodejs/deploy.sh Normal file
View File

@ -0,0 +1,11 @@
#!/bin/sh
echo 'create dir'
if [ ! -d "data" ]; then
mkdir data
fi
echo "copy db"
cp '../../data/ip2region.db' './data/'
echo "npm publish"
npm publish

View File

@ -4,152 +4,335 @@
* project: https://github.com/lionsoul2014/ip2region * project: https://github.com/lionsoul2014/ip2region
* *
* @author dongyado<dongyado@gmail.com> * @author dongyado<dongyado@gmail.com>
* */ * @author leeching<leeching.fx@gmail.com>
var fs = require('fs'); * @author dongwei<maledong_github@outlook.com>
*/
var ipbase = [16777216, 65536, 256, 1]; // for ip2long const fs = require('fs');
var ip2region = {}; const path = require('path');
ip2region.db_file_path = null; const DEFAULT_DB_PATH = path.join(__dirname,'./data/ip2region.db') ;
ip2region.db_fd = null;
var totalBlocks = 0;
var firstIndexPtr = 0;
var lastIndexPtr = 0;
var superBlock = new Buffer(8);
var indexBlockLength = 12;
var totalHeaderLength = 4096;
//#region Private Functions
/**
* Convert ip to long (xxx.xxx.xxx.xxx to a integer)
*
* @param {string} ip
* @return {number} long value
*/
function _ip2long(ip) {
const arr = ip.split('.');
if (arr.length !== 4) {
throw new Error('invalid ip');
}
return arr.reduce((val, n, i) => {
n = Number(n);
if (!Number.isInteger(n) || n < 0 || n > 255) {
throw new Error('invalid ip');
}
return val + IP_BASE[i] * n;
}, 0);
}
/** /**
* binary search synchronized * Get long value from buffer with specified offset
* */ *
ip2region.binarySearchSync = function(ip) * @param {Buffer} buffer
{ * @param {number} offset
var low = 0; * @return {number} long value
var mid = 0; */
var high = totalBlocks; function _getLong(buffer, offset) {
var dataPos = 0; const val =
var pos = 0; (buffer[offset] & 0x000000ff) |
var sip = 0; ((buffer[offset + 1] << 8) & 0x0000ff00) |
var eip = 0; ((buffer[offset + 2] << 16) & 0x00ff0000) |
var indexBuffer = new Buffer(12); ((buffer[offset + 3] << 24) & 0xff000000);
return val < 0 ? val >>> 0 : val;
}
if( typeof(ip) == 'string' ) ip = ip2long(ip); //#endregion
//#region Private Variables
// We don't wanna expose a private global settings to
// the public for safety reason.
const _globalInstances = new Map();
const IP_BASE = [16777216, 65536, 256, 1];
const INDEX_BLOCK_LENGTH = 12;
const TOTAL_HEADER_LENGTH = 8192;
// Private Message Symbols for functions
const PrepareHeader = Symbol('#PrepareHeader');
const CalTotalBlocks = Symbol('#CalsTotalBlocks');
const ReadDataSync = Symbol('#ReadDataSync');
const ReadData = Symbol('#ReadData');
//#endregion
class IP2Region {
//#region Private Functions
[CalTotalBlocks]() {
const superBlock = Buffer.alloc(8);
fs.readSync(this.dbFd, superBlock, 0, 8, 0);
this.firstIndexPtr = _getLong(superBlock, 0);
this.lastIndexPtr = _getLong(superBlock, 4);
this.totalBlocks =
(this.lastIndexPtr - this.firstIndexPtr) / INDEX_BLOCK_LENGTH + 1;
}
[PrepareHeader]() {
fs.readSync(
this.dbFd,
this.headerIndexBuffer,
0,
TOTAL_HEADER_LENGTH,
8
);
for (let i = 0; i < TOTAL_HEADER_LENGTH; i += 8) {
const startIp = _getLong(this.headerIndexBuffer, i);
const dataPtr = _getLong(this.headerIndexBuffer, i + 4);
if (dataPtr == 0) break;
this.headerSip.push(startIp);
this.headerPtr.push(dataPtr);
this.headerLen++; // header index size count
}
}
[ReadData](dataPos, callBack) {
if (dataPos == 0) return callBack(null, null);
const dataLen = (dataPos >> 24) & 0xff;
dataPos = dataPos & 0x00ffffff;
const dataBuffer = Buffer.alloc(dataLen);
fs.read(this.dbFd, dataBuffer, 0, dataLen, dataPos, (err, result) => {
if (err) {
callBack(err, null);
}
else {
const city = _getLong(dataBuffer, 0);
const region = dataBuffer.toString('utf8', 4, dataLen);
callBack(null, { city, region });
}
});
}
[ReadDataSync](dataPos) {
if (dataPos == 0) return null;
const dataLen = (dataPos >> 24) & 0xff;
dataPos = dataPos & 0x00ffffff;
const dataBuffer = Buffer.alloc(dataLen);
fs.readSync(this.dbFd, dataBuffer, 0, dataLen, dataPos);
const city = _getLong(dataBuffer, 0);
const region = dataBuffer.toString('utf8', 4, dataLen);
return { city, region };
}
//#endregion
//#region Static Functions
// Single Instance
static create(dbPath = DEFAULT_DB_PATH) {
let existInstance = _globalInstances.get(dbPath);
if (existInstance == null) {
existInstance = new IP2Region({ dbPath: dbPath });
}
return existInstance;
}
/**
* For backward compatibility
*/
static destroy() {
_globalInstances.forEach(([key, instance]) => {
instance.destroy();
});
}
//#endregion
constructor(options = {}) {
const { dbPath } = options;
// Keep for MemorySearch
this.totalInMemoryBytesSize = fs.statSync(dbPath).size;
this.totalInMemoryBytes = null;
this.dbFd = fs.openSync(dbPath, 'r');
this.dbPath = dbPath;
_globalInstances.set(this.dbPath, this);
this.totalBlocks = this.firstIndexPtr = this.lastIndexPtr = 0;
this[CalTotalBlocks]();
this.headerIndexBuffer = Buffer.alloc(TOTAL_HEADER_LENGTH);
this.headerSip = [];
this.headerPtr = [];
this.headerLen = 0;
this[PrepareHeader]();
}
//#region Public Functions
/**
* Destroy the current file by closing it.
*/
destroy() {
fs.closeSync(this.dbFd);
_globalInstances.delete(this.dbPath);
}
/**
* Sync of binarySearch.
* @param {string} ip The IP address to search for.
* @return {SearchResult} A result something like `{ city: 2163, region: '中国|0|广东省|深圳市|阿里云' }`
*/
binarySearchSync(ip) {
ip = _ip2long(ip);
let low = 0;
let mid = 0;
let high = this.totalBlocks;
let pos = 0;
let sip = 0;
const indexBuffer = Buffer.alloc(12);
// binary search // binary search
while (low <= high) { while (low <= high) {
mid = ((low + high) >> 1); mid = (low + high) >> 1;
pos = firstIndexPtr + mid * indexBlockLength; pos = this.firstIndexPtr + mid * INDEX_BLOCK_LENGTH;
fs.readSync(this.db_fd, indexBuffer, 0, indexBlockLength, pos); fs.readSync(this.dbFd, indexBuffer, 0, INDEX_BLOCK_LENGTH, pos);
sip = getLong(indexBuffer, 0); sip = _getLong(indexBuffer, 0);
//console.log( ' sip : ' + sip + ' eip : ' + eip );
if (ip < sip) { if (ip < sip) {
high = mid - 1; high = mid - 1;
} else { } else {
eip = getLong(indexBuffer, 4); sip = _getLong(indexBuffer, 4);
if ( ip > eip ) { if (ip > sip) {
low = mid + 1; low = mid + 1;
} else { } else {
dataPos = getLong(indexBuffer, 8); sip = _getLong(indexBuffer, 8);
break; break;
} }
} }
} }
return this[ReadDataSync](sip);
// read data
if (dataPos == 0) return null;
var dataLen = ((dataPos >> 24) & 0xFF);
var dataPos = (dataPos & 0x00FFFFFF);
var dataBuffer = new Buffer(dataLen);
fs.readSync(this.db_fd, dataBuffer, 0, dataLen, dataPos);
var city_id = getLong(dataBuffer, 0);
var data = dataBuffer.toString('utf8', 4, dataLen);
//console.log(city_id);
//console.log(data);
return { city: city_id, region: data };
} }
var headerSip = null;
var headerPtr = 0;
var headerLen = 0;
/** /**
* btree search synchronized * Async of binarySearch.
* */ * @param {string} ip The IP address to search for.
ip2region.btreeSearchSync = function(ip) * @param {Function} callBack The callBack function with two parameters, if successful,
{ * err is null and result is `{ city: 2163, region: '中国|0|广东省|深圳市|阿里云' }`
var indexBlockBuffer = new Buffer(indexBlockLength); */
var headerIndexBuffer = new Buffer(totalHeaderLength); binarySearch(ip, callBack) {
if( typeof(ip) == 'string' ) ip = ip2long(ip);
var i = 0; ip = _ip2long(ip);
// header index handler
if (headerSip == null) {
fs.readSync(this.db_fd, headerIndexBuffer, 0, totalHeaderLength, 8);
headerSip = new Array();
headerPtr = new Array();
var startIp = 0; let low = 0;
var dataPtr = 0; let mid = 0;
for ( i = 0; i < totalHeaderLength; i += 8) { let high = this.totalBlocks;
startIp = getLong(headerIndexBuffer, i); let pos = 0;
dataPtr = getLong(headerIndexBuffer, i + 4); let sip = 0;
if ( dataPtr == 0) break; const indexBuffer = Buffer.alloc(12);
const _self = this;
headerSip.push(startIp); // Because `while` is a sync method, we have to convert this to a recursive loop
headerPtr.push(dataPtr); // and in each loop we should continue calling `setImmediate` until we found the IP.
headerLen++; // header index size count function _innerAsyncWhile() {
if (low <= high) {
mid = (low + high) >> 1;
pos = _self.firstIndexPtr + mid * INDEX_BLOCK_LENGTH;
// Now async read the file
fs.read(_self.dbFd, indexBuffer, 0, INDEX_BLOCK_LENGTH, pos, (err) => {
if (err) {
return callBack(err, null);
}
sip = _getLong(indexBuffer, 0);
if (ip < sip) {
high = mid - 1;
setImmediate(_innerAsyncWhile);
} else {
sip = _getLong(indexBuffer, 4);
if (ip > sip) {
low = mid + 1;
setImmediate(_innerAsyncWhile);
} else {
sip = _getLong(indexBuffer, 8);
_self[ReadData](sip, (err, result) => {
callBack(err, result);
});
} }
} }
});
}
}
// Call this immediately
_innerAsyncWhile();
}
/**
* Sync of btreeSearch.
* @param {string} ip The IP address to search for.
* @return {Function} A result something like `{ city: 2163, region: '中国|0|广东省|深圳市|阿里云' }`
*/
btreeSearchSync(ip) {
ip = _ip2long(ip);
// first search (in header index) // first search (in header index)
var low = 0; let low = 0;
var mid = 0; let mid = 0;
var high = headerLen; let high = this.headerLen;
var sptr = 0; let sptr = 0;
var eptr = 0; let eptr = 0;
while (low <= high) { while (low <= high) {
mid = ((low + high) >> 1); mid = (low + high) >> 1;
if (ip == headerSip[mid]) { if (ip == this.headerSip[mid]) {
if ( m > 0) { if (mid > 0) {
sptr = headerPtr[mid - 1]; sptr = this.headerPtr[mid - 1];
eptr = headerPtr[mid]; eptr = this.headerPtr[mid];
} else { } else {
sptr = headerPtr[mid]; sptr = this.headerPtr[mid];
eptr = headerPtr[mid + 1]; eptr = this.headerPtr[mid + 1];
} }
break; break;
} }
if ( ip < headerSip[mid]) { if (ip < this.headerSip[mid]) {
if (mid == 0) { if (mid == 0) {
sptr = headerPtr[mid]; sptr = this.headerPtr[mid];
eptr = headerPtr[mid + 1]; eptr = this.headerPtr[mid + 1];
break; break;
} else if ( ip > headerSip[mid - 1]) { } else if (ip > this.headerSip[mid - 1]) {
sptr = headerPtr[mid - 1]; sptr = this.headerPtr[mid - 1];
eptr = headerPtr[mid]; eptr = this.headerPtr[mid];
break; break;
} }
high = mid - 1; high = mid - 1;
} else { } else {
if ( mid == headerLen - 1) { if (mid == this.headerLen - 1) {
sptr = headerPtr[mid - 1]; sptr = this.headerPtr[mid - 1];
eptr = headerPtr[mid]; eptr = this.headerPtr[mid];
break; break;
} else if ( ip <= headerSip[mid + 1]) { } else if (ip <= this.headerSip[mid + 1]) {
sptr = headerPtr[mid]; sptr = this.headerPtr[mid];
eptr = headerPtr[mid + 1]; eptr = this.headerPtr[mid + 1];
break; break;
} }
low = mid + 1; low = mid + 1;
@ -160,118 +343,292 @@ ip2region.btreeSearchSync = function(ip)
if (sptr == 0) return null; if (sptr == 0) return null;
// second search (in index) // second search (in index)
var blockLen = eptr - sptr; const blockLen = eptr - sptr;
var blockBuffer = new Buffer(blockLen + indexBlockLength); const blockBuffer = Buffer.alloc(blockLen + INDEX_BLOCK_LENGTH);
fs.readSync(this.db_fd, blockBuffer, 0, blockLen + indexBlockLength, sptr); fs.readSync(
this.dbFd,
blockBuffer,
0,
blockLen + INDEX_BLOCK_LENGTH,
sptr
);
low = 0; low = 0;
high = blockLen / indexBlockLength; high = blockLen / INDEX_BLOCK_LENGTH;
var p = 0; let p = 0;
var sip = 0; let sip = 0;
var eip = 0;
var dataPtr = 0;
while (low <= high) { while (low <= high) {
mid = ((low + high) >> 1); mid = (low + high) >> 1;
p = mid * indexBlockLength; p = mid * INDEX_BLOCK_LENGTH;
sip = getLong(blockBuffer, p); sip = _getLong(blockBuffer, p);
if (ip < sip) { if (ip < sip) {
high = mid - 1; high = mid - 1;
} else { } else {
eip = getLong(blockBuffer, p + 4); sip = _getLong(blockBuffer, p + 4);
if (ip > eip) { if (ip > sip) {
low = mid + 1; low = mid + 1;
} else { } else {
dataPtr = getLong(blockBuffer, p + 8); sip = _getLong(blockBuffer, p + 8);
break; break;
} }
} }
} }
return this[ReadDataSync](sip);
// read data
if (dataPtr == 0) return null;
var dataLen = ((dataPtr >> 24) & 0xFF);
var dataPtr = (dataPtr & 0x00FFFFFF);
var dataBuffer = new Buffer(dataLen);
fs.readSync(this.db_fd, dataBuffer, 0, dataLen, dataPtr);
var city_id = getLong(dataBuffer, 0);
var data = dataBuffer.toString('utf8', 4, dataLen);
//console.log(city_id);
//console.log(data);
return { city: city_id, region: data };
} }
/** /**
* convert ip to long (xxx.xxx.xxx.xxx to a integer) * Async of btreeSearch.
* */ * @param {string} ip The IP address to search for.
function ip2long(ip) * @param {Function} callBack The callBack function with two parameters, if successful,
{ * err is null and result is `{ city: 2163, region: '中国|0|广东省|深圳市|阿里云' }`
var val = 0; */
ip.split('.').forEach(function(ele, i){ btreeSearch(ip, callBack) {
val += ipbase[i] * ele; ip = _ip2long(ip);
// first search (in header index)
let low = 0;
let mid = 0;
let high = this.headerLen;
let sptr = 0;
let eptr = 0;
while (low <= high) {
mid = (low + high) >> 1;
if (ip == this.headerSip[mid]) {
if (mid > 0) {
sptr = this.headerPtr[mid - 1];
eptr = this.headerPtr[mid];
} else {
sptr = this.headerPtr[mid];
eptr = this.headerPtr[mid + 1];
}
break;
}
if (ip < this.headerSip[mid]) {
if (mid == 0) {
sptr = this.headerPtr[mid];
eptr = this.headerPtr[mid + 1];
break;
} else if (ip > this.headerSip[mid - 1]) {
sptr = this.headerPtr[mid - 1];
eptr = this.headerPtr[mid];
break;
}
high = mid - 1;
} else {
if (mid == this.headerLen - 1) {
sptr = this.headerPtr[mid - 1];
eptr = this.headerPtr[mid];
break;
} else if (ip <= this.headerSip[mid + 1]) {
sptr = this.headerPtr[mid];
eptr = this.headerPtr[mid + 1];
break;
}
low = mid + 1;
}
}
// match nothing
if (sptr == 0) return callBack(null, null);
let p = 0;
let sip = 0;
// second search (in index)
const blockLen = eptr - sptr;
const blockBuffer = Buffer.alloc(blockLen + INDEX_BLOCK_LENGTH);
low = 0;
high = blockLen / INDEX_BLOCK_LENGTH;
const _self = this;
function _innerAsyncWhile() {
if (low <= high) {
mid = (low + high) >> 1;
p = mid * INDEX_BLOCK_LENGTH;
// Use this to call the method itself as
// an asynchronize step
fs.read(_self.dbFd, blockBuffer,
0,
blockLen + INDEX_BLOCK_LENGTH,
sptr, (err) => {
if (err) {
return callBack(err, null);
}
sip = _getLong(blockBuffer, p);
if (ip < sip) {
high = mid - 1;
setImmediate(_innerAsyncWhile);
} else {
sip = _getLong(blockBuffer, p + 4);
if (ip > sip) {
low = mid + 1;
setImmediate(_innerAsyncWhile);
} else {
sip = _getLong(blockBuffer, p + 8);
_self[ReadData](sip, (err, result) => {
callBack(err, result);
}); });
}
return val; }
});
}
else {
// If we found nothing, return null
return callBack(null, null);
}
} }
_innerAsyncWhile();
}
/** /**
* get long value from buffer with specified offset * Sync of MemorySearch.
* */ * @param {String} ip
function getLong(buffer, offset) */
{ memorySearchSync(ip) {
var val = (
(buffer[offset] & 0x000000FF) |
((buffer[offset + 1] << 8) & 0x0000FF00) |
((buffer[offset + 2] << 16) & 0x00FF0000) |
((buffer[offset + 3] << 24) & 0xFF000000)
);
// convert to unsigned int ip = _ip2long(ip);
if (val < 0) {
val = val >>> 0; if (this.totalInMemoryBytes === null) {
}
return val; this.totalInMemoryBytes = Buffer.alloc(this.totalInMemoryBytesSize);
fs.readSync(this.dbFd, this.totalInMemoryBytes, 0, this.totalInMemoryBytesSize, 0);
this.firstIndexPtr = _getLong(this.totalInMemoryBytes, 0);
this.lastIndexPtr = _getLong(this.totalInMemoryBytes, 4);
this.totalBlocks = ((this.lastIndexPtr - this.firstIndexPtr) / INDEX_BLOCK_LENGTH) | 0 + 1;
} }
let l = 0, h = this.totalBlocks;
let sip = 0;
let m = 0, p = 0;
exports.create = function(db_path) while (l <= h) {
{ m = (l + h) >> 1;
if (typeof(db_path) == "undefined" || fs.exists(db_path) ) { p = (this.firstIndexPtr + m * INDEX_BLOCK_LENGTH) | 0;
throw("[ip2region] db file not exists : " + db_path);
sip = _getLong(this.totalInMemoryBytes, p);
if (ip < sip) {
h = m - 1;
}
else {
sip = _getLong(this.totalInMemoryBytes, p + 4);
if (ip > sip) {
l = m + 1;
}
else {
sip = _getLong(this.totalInMemoryBytes, p + 8);
//not matched
if (sip === 0) return null;
//get the data
let dataLen = ((sip >> 24) & 0xFF) | 0;
let dataPtr = ((sip & 0x00FFFFFF)) | 0;
let city = _getLong(this.totalInMemoryBytes, dataPtr);
const bufArray = new Array();
for (let startPos = dataPtr + 4, i = startPos; i < startPos + dataLen - 4; ++i) {
bufArray.push(this.totalInMemoryBytes[i]);
}
const region = Buffer.from(bufArray, 0).toString();
return { city, region };
}
}
}
} }
ip2region.db_file_path = db_path; /**
* Async of MemorySearch.
* @param {String} ip
*/
memorySearch(ip, callBack) {
try { let _ip = _ip2long(ip);
ip2region.db_fd = fs.openSync(ip2region.db_file_path, 'r'); let l = 0, h = this.totalBlocks;
} catch(e) { let sip = 0;
throw("[ip2region] Can not open ip2region.db file , path : " let m = 0, p = 0;
+ ip2region.db_file_path); let self = this;
function _innerMemorySearchLoop() {
if (l <= h) {
m = (l + h) >> 1;
p = (self.firstIndexPtr + m * INDEX_BLOCK_LENGTH) | 0;
sip = _getLong(self.totalInMemoryBytes, p);
if (_ip < sip) {
h = m - 1;
setImmediate(_innerMemorySearchLoop);
}
else {
sip = _getLong(self.totalInMemoryBytes, p + 4);
if (_ip > sip) {
l = m + 1;
setImmediate(_innerMemorySearchLoop);
}
else {
sip = _getLong(self.totalInMemoryBytes, p + 8);
//not matched
if (sip === 0) return callBack(null, null);
//get the data
let dataLen = ((sip >> 24) & 0xFF) | 0;
let dataPtr = ((sip & 0x00FFFFFF)) | 0;
let city = _getLong(self.totalInMemoryBytes, dataPtr);
const bufArray = new Array();
for (let startPos = dataPtr + 4, i = startPos; i < startPos + dataLen - 4; ++i) {
bufArray.push(self.totalInMemoryBytes[i]);
}
const region = Buffer.from(bufArray).toString();
callBack(null, { city, region });
}
}
}
else {
callBack(null, null);
}
} }
// init basic search environment if (this.totalInMemoryBytes === null) {
if (totalBlocks == 0) {
fs.readSync(ip2region.db_fd, superBlock, 0, 8, 0);
firstIndexPtr = getLong(superBlock, 0);
lastIndexPtr = getLong(superBlock, 4);
totalBlocks = (lastIndexPtr - firstIndexPtr)
/ indexBlockLength + 1;
}
return ip2region; this.totalInMemoryBytes = Buffer.alloc(this.totalInMemoryBytesSize);
}
fs.read(this.dbFd, this.totalInMemoryBytes, 0, this.totalInMemoryBytesSize, 0, (err) => {
exports.destroy = function(ip2rObj) if (err) {
{ callBack(err, null);
ip2rObj.db_file_path = null;
fs.closeSync(ip2rObj.db_fd);
} }
else {
this.firstIndexPtr = _getLong(this.totalInMemoryBytes, 0);
this.lastIndexPtr = _getLong(this.totalInMemoryBytes, 4);
this.totalBlocks = ((this.lastIndexPtr - this.firstIndexPtr) / INDEX_BLOCK_LENGTH) | 0 + 1;
_innerMemorySearchLoop();
}
});
}
else {
_innerMemorySearchLoop();
}
}
}
//#endregion
module.exports = IP2Region;

4049
binding/nodejs/package-lock.json generated Normal file

File diff suppressed because it is too large Load Diff

View File

@ -1,11 +1,13 @@
{ {
"name": "ip2region", "name": "lionsoul-ip2region",
"version": "0.0.1", "version": "0.0.1",
"description": "ip database ", "description": "ip database ",
"main": "ip2region.js", "main": "ip2region.js",
"scripts": { "scripts": {
"test": "echo \"Error: no test specified\" && exit 1" "test": "jest",
"coverage": "npm run test && jest --coverage"
}, },
"files": ["data/"],
"repository": { "repository": {
"type": "git", "type": "git",
"url": "https://github.com/lionsoul2016/ip2region" "url": "https://github.com/lionsoul2016/ip2region"
@ -20,5 +22,9 @@
"bugs": { "bugs": {
"url": "https://github.com/lionsoul2016/ip2region/issues" "url": "https://github.com/lionsoul2016/ip2region/issues"
}, },
"homepage": "https://github.com/lionsoul2016/ip2region" "homepage": "https://github.com/lionsoul2016/ip2region",
"devDependencies": {
"jest": "^19.0.2",
"benchmark": "^2.1.4"
}
} }

View File

@ -1,15 +0,0 @@
/**
* test program
* */
var Ip2region = require('./ip2region');
// 1. create a ip2region object
var ipObj = Ip2region.create('../../data/ip2region.db');
// 2. use
//console.log(ipObj.binarySearchSync("120.24.78.68"));
console.log(ipObj.btreeSearchSync("120.24.78.68"));
// 3. destroy
Ip2region.destroy(ipObj);

View File

@ -0,0 +1,94 @@
const Benchmark = require('benchmark');
const suite = new Benchmark.Suite();
const searcher = require('../../ip2region').create('../../data/ip2region.db');
const testDatas = require('../utils/testData');
const asyncFor = require('../utils/asyncFor');
suite.add("MemorySearchSync", () => {
for (let i = 0; i < testDatas.length; ++i) {
searcher.memorySearchSync(testDatas[i]);
}
})
.add("BinarySearchSync", () => {
for (let i = 0; i < testDatas.length; ++i) {
searcher.binarySearchSync(testDatas[i]);
}
})
.add("BtreeSearchSync", () => {
for (let i = 0; i < testDatas.length; ++i) {
searcher.btreeSearchSync(testDatas[i]);
}
})
.add("MemorySearch", {
defer: true,
fn: function (completeCallBack) {
asyncFor(testDatas,
(v, c) => {
searcher.memorySearch(v, () => {
c();
});
},
() => {
completeCallBack.resolve();
});
}
})
.add("BinarySearch", {
defer: true,
fn: function (completeCallBack) {
asyncFor(testDatas,
(v, c) => {
searcher.binarySearch(v, () => {
c();
});
},
() => {
completeCallBack.resolve();
});
}
})
.add("BtreeSearch", {
defer: true,
fn: function (completeCallBack) {
asyncFor(testDatas,
(v, c) => {
searcher.btreeSearch(v, () => {
c();
});
},
() => {
completeCallBack.resolve();
});
}
})
.on('cycle', function (event) {
console.log(String(event.target));
})
.on('complete', function () {
let results = new Array();
for (let prop in this) {
if (!isNaN(prop)) {
const eachResult = {
name: this[prop].name,
mean: this[prop].stats.mean * 1000, //second => millisecond
moe: this[prop].stats.moe,
rme: this[prop].stats.rme,
sem: this[prop].stats.sem
}
results.push(eachResult);
}
}
results = results.sort((a, b) => { return a.mean - b.mean });
console.log(`Rand\t${'Name'.padEnd(20)}Time (in milliseconds)`);
let id = 1;
for (let r of results) {
console.log(`${id++}\t${r.name.padEnd(20)}${r.mean.toFixed(3)}`);
}
})
.run({ async: true });

View File

@ -0,0 +1,145 @@
// This test is used for tesing of a static function `create` of IP2Region
const IP2Region = require('../../ip2region');
const testIps = require('../utils/testData');
const asyncFor = require('../utils/asyncFor');
describe('Constructor Test', () => {
let instance;
beforeAll(() => {
instance = new IP2Region({ dbPath: '../../data/ip2region.db' });
});
afterAll(() => {
IP2Region.destroy();
});
test('btreeSearchSync query', () => {
for (const ip of testIps) {
expect(instance.btreeSearchSync(ip)).toMatchSnapshot();
}
});
test('binarySearchSync query', () => {
for (const ip of testIps) {
expect(instance.binarySearchSync(ip)).toMatchSnapshot();
}
});
test('memorySearchSync query', () => {
for (const ip of testIps) {
expect(instance.memorySearchSync(ip)).toMatchSnapshot();
}
});
//#region callBack
test('binarySearch query', (done) => {
asyncFor(testIps,
(value, continueCallBack) => {
instance.binarySearch(value, (err, result) => {
expect(err).toBe(null);
expect(result).toMatchSnapshot();
continueCallBack();
});
},
() => { done() });
});
test('btreeSearch query', (done) => {
asyncFor(testIps,
(value, continueCallBack) => {
instance.btreeSearch(value, (err, result) => {
expect(err).toBe(null);
expect(result).toMatchSnapshot();
continueCallBack();
});
},
() => { done() });
});
test('memorySearch query', (done) => {
asyncFor(testIps,
(value, continueCallBack) => {
instance.memorySearch(value, (err, result) => {
expect(err).toBe(null);
expect(result).toMatchSnapshot();
continueCallBack();
});
},
() => { done() });
});
//#endregion
//#region Async Promisify test
const node_ver = require('../utils/fetchMainVersion');
// If we have Nodejs >= 8, we now support `async` and `await`
if (node_ver >= 8) {
const asyncBinarySearch = async (ip) => {
return new Promise((resolve, reject) => {
instance.binarySearch(ip, (err, result) => {
if (err) {
reject(err);
}
else {
resolve(result);
}
});
});
};
const asyncBtreeSearch = async (ip) => {
return new Promise((resolve, reject) => {
instance.btreeSearch(ip, (err, result) => {
if (err) {
reject(err);
}
else {
resolve(result);
}
});
});
};
const asyncMemorySearch = async (ip) => {
return new Promise((succ, fail) => {
instance.memorySearch(ip, (err, result) => {
if (err) {
fail(err);
}
else {
succ(result);
}
});
});
}
test('async binarySearch query', async () => {
for (let i = 0; i < testIps.length; ++i) {
const result = await asyncBinarySearch(testIps[i]);
expect(result).toMatchSnapshot();
}
});
test('async btreeSearch query', async () => {
for (let i = 0; i < testIps.length; ++i) {
const result = await asyncBtreeSearch(testIps[i]);
expect(result).toMatchSnapshot();
}
});
test('async memorySearch query', async () => {
for (let i = 0; i < testIps.length; ++i) {
const result = await asyncMemorySearch(testIps[i]);
expect(result).toMatchSnapshot();
}
});
}
//#endregion
});

View File

@ -0,0 +1,146 @@
// This test is used for tesing of a static function `create` of IP2Region
const IP2Region = require('../../ip2region');
const testIps = require('../utils/testData');
const asyncFor = require('../utils/asyncFor');
describe('Create Test', () => {
let instance;
beforeAll(() => {
instance = IP2Region.create('../../data/ip2region.db');
});
afterAll(() => {
IP2Region.destroy();
});
test('btreeSearchSync query', () => {
for (const ip of testIps) {
expect(instance.btreeSearchSync(ip)).toMatchSnapshot();
}
});
test('binarySearchSync query', () => {
for (const ip of testIps) {
expect(instance.binarySearchSync(ip)).toMatchSnapshot();
}
});
test('memorySearchSync query', () => {
for (const ip of testIps) {
expect(instance.memorySearchSync(ip)).toMatchSnapshot();
}
});
//#region callBack
test('binarySearch query', (done) => {
asyncFor(testIps,
(value, continueCallBack) => {
instance.binarySearch(value, (err, result) => {
expect(err).toBe(null);
expect(result).toMatchSnapshot();
continueCallBack();
});
},
() => { done() });
});
test('btreeSearch query', (done) => {
asyncFor(testIps,
(value, continueCallBack) => {
instance.btreeSearch(value, (err, result) => {
expect(err).toBe(null);
expect(result).toMatchSnapshot();
continueCallBack();
});
},
() => { done() });
});
test('memorySearch query', (done) => {
asyncFor(testIps,
(value, continueCallBack) => {
instance.memorySearch(value, (err, result) => {
expect(err).toBe(null);
expect(result).toMatchSnapshot();
continueCallBack();
});
},
() => { done() });
});
//#endregion
//#region Async Promisify test
const node_ver = require('../utils/fetchMainVersion');
// If we have Nodejs >= 8, we now support `async` and `await`
if (node_ver >= 8) {
const asyncBinarySearch = async (ip) => {
return new Promise((resolve, reject) => {
instance.binarySearch(ip, (err, result) => {
if (err) {
reject(err);
}
else {
resolve(result);
}
});
});
};
const asyncBtreeSearch = async (ip) => {
return new Promise((resolve, reject) => {
instance.btreeSearch(ip, (err, result) => {
if (err) {
reject(err);
}
else {
resolve(result);
}
});
});
};
const asyncMemorySearch = async (ip) => {
return new Promise((succ, fail) => {
instance.memorySearch(ip, (err, result) => {
if (err) {
fail(err);
}
else {
succ(result);
}
});
});
}
test('async binarySearch query', async () => {
for (let i = 0; i < testIps.length; ++i) {
const result = await asyncBinarySearch(testIps[i]);
expect(result).toMatchSnapshot();
}
});
test('async btreeSearch query', async () => {
for (let i = 0; i < testIps.length; ++i) {
const result = await asyncBtreeSearch(testIps[i]);
expect(result).toMatchSnapshot();
}
});
test('async memorySearch query', async () => {
for (let i = 0; i < testIps.length; ++i) {
const result = await asyncMemorySearch(testIps[i]);
expect(result).toMatchSnapshot();
}
});
}
//#endregion
});

View File

@ -0,0 +1,28 @@
// This test is used for tesing of exceptions
const IP2Region = require('../../ip2region');
describe('Constructor Test', () => {
let instance;
beforeAll(() => {
instance = new IP2Region({ dbPath: '../../data/ip2region.db' })
});
afterAll(() => {
instance.destroy();
});
test('IP invalid test', () => {
const invalidIps = ['255.234.233', '255.255.-1.255', null, undefined, '', 'x.255.y.200'];
for (const ip of invalidIps) {
expect(() => instance.btreeSearchSync(ip)).toThrow();
expect(() => instance.binarySearchSync(ip)).toThrow();
}
});
test('File Not Found test', () => {
expect(() => new IP2Region({ dbPath: 'A Bad File or Path Here' })).toThrow();
});
});

View File

@ -0,0 +1,23 @@
/**
* Async For
* @param {Array} groupArray
* @param {Function} exeCallBack
* @param {Function} finalCallBack
*/
function asyncFor(groupArray, exeCallBack, finalCallBack) {
let i = 0;
function _innerAsyncLoop() {
if (i < groupArray.length) {
exeCallBack(groupArray[i++], _innerAsyncLoop);
}
else {
finalCallBack();
}
}
_innerAsyncLoop();
}
module.exports = asyncFor;

View File

@ -0,0 +1,9 @@
let node_ver = process.version
// Because nodejs's version is something like `v8.11.3`. So we should ignore `v` first
node_ver = node_ver.substr(1);
// Splitted by `.`
node_ver = node_ver.split('.');
// Take the main version number
node_ver = parseInt(node_ver[0]);
module.exports = node_ver;

View File

@ -0,0 +1,16 @@
module.exports = [
'0.0.0.0',
'10.10.10.10',
'210.109.255.230',
'192.168.0.1',
'255.255.255.255',
'77.49.66.88',
'210.248.255.231',
'35.193.251.120',
'197.84.60.202',
'183.196.233.159',
'20.108.91.101',
'120.196.148.137',
'249.255.250.200',
'112.65.1.130'
]

1979
binding/nodejs/yarn.lock Normal file

File diff suppressed because it is too large Load Diff

View File

@ -7,7 +7,7 @@
*/ */
defined('INDEX_BLOCK_LENGTH') or define('INDEX_BLOCK_LENGTH', 12); defined('INDEX_BLOCK_LENGTH') or define('INDEX_BLOCK_LENGTH', 12);
defined('TOTAL_HEADER_LENGTH') or define('TOTAL_HEADER_LENGTH', 4096); defined('TOTAL_HEADER_LENGTH') or define('TOTAL_HEADER_LENGTH', 8192);
class Ip2Region class Ip2Region
{ {
@ -106,7 +106,7 @@ class Ip2Region
} }
/** /**
* get the data block throught the specifield ip address or long ip numeric with binary search algorithm * get the data block through the specified ip address or long ip numeric with binary search algorithm
* *
* @param ip * @param ip
* @return mixed Array or NULL for any error * @return mixed Array or NULL for any error
@ -174,7 +174,7 @@ class Ip2Region
} }
/** /**
* get the data block associated with the specifield ip with b-tree search algorithm * get the data block associated with the specified ip with b-tree search algorithm
* @Note: not thread safe * @Note: not thread safe
* *
* @param ip * @param ip
@ -266,7 +266,7 @@ class Ip2Region
fseek($this->dbFileHandler, $sptr); fseek($this->dbFileHandler, $sptr);
$index = fread($this->dbFileHandler, $blockLen + INDEX_BLOCK_LENGTH); $index = fread($this->dbFileHandler, $blockLen + INDEX_BLOCK_LENGTH);
$dataptr = 0; $dataPtr = 0;
$l = 0; $h = $blockLen / INDEX_BLOCK_LENGTH; $l = 0; $h = $blockLen / INDEX_BLOCK_LENGTH;
while ( $l <= $h ) { while ( $l <= $h ) {
$m = (($l + $h) >> 1); $m = (($l + $h) >> 1);
@ -279,18 +279,18 @@ class Ip2Region
if ( $ip > $eip ) { if ( $ip > $eip ) {
$l = $m + 1; $l = $m + 1;
} else { } else {
$dataptr = self::getLong($index, $p + 8); $dataPtr = self::getLong($index, $p + 8);
break; break;
} }
} }
} }
//not matched //not matched
if ( $dataptr == 0 ) return NULL; if ( $dataPtr == 0 ) return NULL;
//3. get the data //3. get the data
$dataLen = (($dataptr >> 24) & 0xFF); $dataLen = (($dataPtr >> 24) & 0xFF);
$dataPtr = ($dataptr & 0x00FFFFFF); $dataPtr = ($dataPtr & 0x00FFFFFF);
fseek($this->dbFileHandler, $dataPtr); fseek($this->dbFileHandler, $dataPtr);
$data = fread($this->dbFileHandler, $dataLen); $data = fread($this->dbFileHandler, $dataLen);
@ -358,4 +358,3 @@ class Ip2Region
$this->HeaderPtr = NULL; $this->HeaderPtr = NULL;
} }
} }
?>

View File

@ -8,7 +8,7 @@
if ( $argc < 2 ) { if ( $argc < 2 ) {
$usage = <<<EOF $usage = <<<EOF
Usage: php Test.php [ip2region db file] [alrogrithm] Usage: php Test.php [ip2region db file] [alrogrithm]
+-Algorithm: binary or b-tree\n +-Optional Algorithm: binary, b-tree, memory\n
EOF; EOF;
exit($usage); exit($usage);
} }

View File

@ -1,10 +1,10 @@
# ip2region 的PHP拓展(php5版本) # ip2region client - PHP扩展(php5版本)
### 安装步骤 ### 安装步骤
* git clone https://github.com/lionsoul2016/ip2region.git * git clone https://github.com/lionsoul2014/ip2region.git
* cd ip2region * cd ip2region
* cp binding/php_extension/php5/ip2region 到 php source code 的 ext 目录下 * cp binding/php_extension/php5/ip2region 到 php source code 的 ext 目录下
* cp binding/c/下面所有的文件到 php source code 的ext/ip2region/lib 目录下 * cp binding/c/ 里面所有的文件到 php source code 的 ext/ip2region/lib 目录下
* 在 ext/ip2region 下,运行 * 在 ext/ip2region 下,运行
phpize phpize
@ -16,7 +16,7 @@
extension=ip2region.so extension=ip2region.so
ip2region.db_file=/path/to/ip2region.db ip2region.db_file=/path/to/ip2region.db
ip2region.db 在项目根目录下的data文件夹下 默认 ip2region.db 在项目根目录下的 data 文件夹下,如有改动,请修改 ip2region.ini 的 db_file
* 测试 * 测试
@ -26,4 +26,4 @@ ip2region.db 在项目根目录下的data文件夹下
### 使用 ### 使用
参考ip2region/ip2region.php 参考当前目录下的 ip2region.php

View File

@ -67,10 +67,10 @@ void search(
if ( res == 1 ) if ( res == 1 )
{ {
add_assoc_long( *return_value, "cityId", (*_block).city_id); add_assoc_long( *return_value, "city_id", (*_block).city_id);
add_assoc_string( *return_value, "region", (*_block).region, 1); add_assoc_string( *return_value, "region", (*_block).region, 1);
} else { } else {
add_assoc_long( *return_value, "cityId", 0); add_assoc_long( *return_value, "city_id", 0);
add_assoc_string( *return_value, "region", "[Error] Search Failed! Please check the path of ip2region db file.", 1); add_assoc_string( *return_value, "region", "[Error] Search Failed! Please check the path of ip2region db file.", 1);
} }
} }

View File

@ -1,10 +1,10 @@
# ip2region 的PHP拓展(php7版本) # ip2region client - PHP扩展(php7版本)
### 安装步骤 ### 安装步骤
* git clone https://github.com/lionsoul2016/ip2region.git * git clone https://github.com/lionsoul2014/ip2region.git
* cd ip2region * cd ip2region
* cp binding/php_extension/php5/ip2region 到 php source code 的ext目录下 * cp binding/php_extension/php7/ip2region 到 php source code 的 ext 目录下
* cp binding/c/下面所有的文件到 php source code 的ext/ip2region/lib 目录下 * cp binding/c/ 里面所有的文件到 php source code 的 ext/ip2region/lib 目录下
* 在 ext/ip2region 下,运行 * 在 ext/ip2region 下,运行
phpize phpize
@ -16,7 +16,7 @@
extension=ip2region.so extension=ip2region.so
ip2region.db_file=/path/to/ip2region.db ip2region.db_file=/path/to/ip2region.db
ip2region.db 在项目根目录下的data文件夹下 默认 ip2region.db 在项目根目录下的 data 文件夹下,如有改动,请修改 ip2region.ini 的 db_file
* 测试 * 测试
@ -26,4 +26,4 @@ ip2region.db 在项目根目录下的data文件夹下
### 使用 ### 使用
参考ip2region/ip2region.php 参考当前目录下的 ip2region.php

View File

@ -67,12 +67,12 @@ void search(
if ( res == 1 ) if ( res == 1 )
{ {
add_assoc_long( *return_value, "cityId", (*_block).city_id); add_assoc_long( *return_value, "city_id", (*_block).city_id);
//add_assoc_string( *return_value, "region", (*_block).region, 1); //add_assoc_string( *return_value, "region", (*_block).region, 1);
// for phpng // for phpng
add_assoc_string( *return_value, "region", (*_block).region); add_assoc_string( *return_value, "region", (*_block).region);
} else { } else {
add_assoc_long( *return_value, "cityId", 0); add_assoc_long( *return_value, "city_id", 0);
//add_assoc_string( *return_value, "region", "[Error] Search Failed! Please check the path of ip2region db file.", 1); //add_assoc_string( *return_value, "region", "[Error] Search Failed! Please check the path of ip2region db file.", 1);
// for phpng // for phpng
add_assoc_string( *return_value, "region", "[Error] Search Failed! Please check the path of ip2region db file."); add_assoc_string( *return_value, "region", "[Error] Search Failed! Please check the path of ip2region db file.");

View File

@ -0,0 +1,54 @@
#-*- coding:utf-8 -*-
"""
" ip2region python seacher client module benchmark test file
"
" Autho: koma<komazhang@foxmail.com>
" Date : 2018-10-04
"""
import threading
import time, sys
from ip2Region import Ip2Region
class BenchmarkThread(threading.Thread):
__searcher = None
__lock = None
def __init__(self, searcher, lock):
self.__searcher = searcher
self.__lock = lock
threading.Thread.__init__(self)
def run(self):
self.__lock.acquire()
try:
sTime = time.time() * 1000
data = self.__searcher.memorySearch("49.220.138.233")
eTime = time.time() * 1000
# @Note uncomment the print to make it more like the product environment
print("%s|%s in %5f millseconds" % (data["city_id"], data["region"].decode('utf-8'), eTime - sTime))
finally:
self.__lock.release()
if __name__ == "__main__":
dbFile = "./data/ip2region.db"
if ( len(sys.argv) > 2 ):
dbFile = sys.argv[1];
threads = []
searcher = Ip2Region(dbFile)
lock = threading.Lock()
for i in range(10000):
t = BenchmarkThread(searcher, lock)
threads.append(t)
sTime = time.time() * 1000
for t in threads:
t.start()
for t in threads:
t.join()
eTime = time.time() * 1000
print("Benchmark done: %5f" % (eTime - sTime))

View File

@ -8,11 +8,16 @@
import struct, io, socket, sys import struct, io, socket, sys
class Ip2Region(object): class Ip2Region(object):
__INDEX_BLOCK_LENGTH = 12
__TOTAL_HEADER_LENGTH = 8192
__f = None
__headerSip = [] __headerSip = []
__headerPtr = [] __headerPtr = []
__f = None __headerLen = 0
__sPtr = 0 __indexSPtr = 0
__indexLen = 0 __indexLPtr = 0
__indexCount = 0
__dbBinStr = '' __dbBinStr = ''
def __init__(self, dbfile): def __init__(self, dbfile):
@ -27,34 +32,29 @@ class Ip2Region(object):
if self.__dbBinStr == '': if self.__dbBinStr == '':
self.__dbBinStr = self.__f.read() #read all the contents in file self.__dbBinStr = self.__f.read() #read all the contents in file
self.__sPtr = self.getLong(self.__dbBinStr, 0) self.__indexSPtr = self.getLong(self.__dbBinStr, 0)
endPtr = self.getLong(self.__dbBinStr, 4) self.__indexLPtr = self.getLong(self.__dbBinStr, 4)
self.__indexLen = endPtr - self.__sPtr self.__indexCount = int((self.__indexLPtr - self.__indexSPtr)/self.__INDEX_BLOCK_LENGTH)+1
startPtr = self.__sPtr l, h, dataPtr = (0, self.__indexCount, 0)
indexLen = self.__indexLen
dbBinStr = self.__dbBinStr
l, h, mixPtr = (0, int(indexLen/12), 0)
while l <= h: while l <= h:
m = int((l+h)/2) m = int((l+h) >> 1)
ptr = startPtr + m*12 p = self.__indexSPtr + m*self.__INDEX_BLOCK_LENGTH
sip = self.getLong(self.__dbBinStr, p)
sip = self.getLong(dbBinStr, ptr) if ip < sip:
eip = self.getLong(dbBinStr, ptr+4)
if ip >= sip:
if ip > eip:
l = m + 1
else:
mixPtr = self.getLong(dbBinStr, ptr+8)
break;
else:
h = m -1 h = m -1
else:
eip = self.getLong(self.__dbBinStr, p+4)
if ip > eip:
l = m + 1;
else:
dataPtr = self.getLong(self.__dbBinStr, p+8)
break
if mixPtr == 0: return "N2" if dataPtr == 0: raise Exception("Data pointer not found")
return self.returnData(mixPtr) return self.returnData(dataPtr)
def binarySearch(self, ip): def binarySearch(self, ip):
""" """
@ -63,41 +63,34 @@ class Ip2Region(object):
""" """
if not ip.isdigit(): ip = self.ip2long(ip) if not ip.isdigit(): ip = self.ip2long(ip)
if self.__indexLen < 1: if self.__indexCount == 0:
self.__f.seek(0) self.__f.seek(0)
b = self.__f.read(8) superBlock = self.__f.read(8)
self.__sPtr = self.getLong(b, 0) self.__indexSPtr = self.getLong(superBlock, 0)
endPtr = self.getLong(b, 4) self.__indexLPtr = self.getLong(superBlock, 4)
self.__indexLen = endPtr - self.__sPtr self.__indexCount = int((self.__indexLPtr - self.__indexSPtr) / self.__INDEX_BLOCK_LENGTH) + 1
startPtr = self.__sPtr l, h, dataPtr = (0, self.__indexCount, 0)
indexLen = self.__indexLen
self.__f.seek(startPtr)
b = self.__f.read(indexLen+12)
l, h, mixPtr = (0, int(indexLen/12), 0)
while l <= h: while l <= h:
m = int((l+h)/2) m = int((l+h) >> 1)
ptr = startPtr + m*12 p = m*self.__INDEX_BLOCK_LENGTH
self.__f.seek(ptr)
b = self.__f.read(12) self.__f.seek(self.__indexSPtr+p)
sip = self.getLong(b, 0) buffer = self.__f.read(self.__INDEX_BLOCK_LENGTH)
eip = self.getLong(b, 4) sip = self.getLong(buffer, 0)
if ip < sip:
if ip >= sip: h = m - 1
else:
eip = self.getLong(buffer, 4)
if ip > eip: if ip > eip:
l = m + 1 l = m + 1
else: else:
mixPtr = self.getLong(b, 8) dataPtr = self.getLong(buffer, 8)
break; break
else:
h = m - 1
if mixPtr == 0: return "N2" if dataPtr == 0: raise Exception("Data pointer not found")
return self.returnData(mixPtr) return self.returnData(dataPtr)
def btreeSearch(self, ip): def btreeSearch(self, ip):
""" """
@ -107,82 +100,81 @@ class Ip2Region(object):
if not ip.isdigit(): ip = self.ip2long(ip) if not ip.isdigit(): ip = self.ip2long(ip)
if len(self.__headerSip) < 1: if len(self.__headerSip) < 1:
headerLen = 0
#pass the super block #pass the super block
self.__f.seek(8) self.__f.seek(8)
#read the header block #read the header block
b = self.__f.read(4086) b = self.__f.read(self.__TOTAL_HEADER_LENGTH)
#parse the header block #parse the header block
sip = None for i in range(0, len(b), 8):
ptr = None
for i in range(0, len(b)-1, 8):
sip = self.getLong(b, i) sip = self.getLong(b, i)
ptr = self.getLong(b, i+4) ptr = self.getLong(b, i+4)
if ptr == 0: if ptr == 0:
break break
self.__headerSip.append(sip) self.__headerSip.append(sip)
self.__headerPtr.append(ptr) self.__headerPtr.append(ptr)
headerLen += 1
self.__headerLen = headerLen
headerLen = len(self.__headerSip) - 1 l, h, sptr, eptr = (0, self.__headerLen, 0, 0)
l, h, sptr, eptr = (0, headerLen, 0, 0)
while l <= h: while l <= h:
m = int((l+h)/2) m = int((l+h) >> 1)
if ip == self.__headerSip[m]: if ip == self.__headerSip[m]:
if m > 0: if m > 0:
sptr = self.__headerPtr[m-1] sptr = self.__headerPtr[m-1]
eptr = self.__headerPtr[m] eptr = self.__headerPtr[m]
break;
else: else:
sptr = self.__headerPtr[m] sptr = self.__headerPtr[m]
eptr = self.__headerPtr[m+1] eptr = self.__headerPtr[m+1]
break; break
if ip > self.__headerSip[m]: if ip < self.__headerSip[m]:
if m == headerLen:
sptr = self.__headerPtr[m-1]
eptr = self.__headerPtr[m]
break;
elif ip < self.__headerSip[m+1]:
sptr = self.__headerPtr[m]
eptr = self.__headerPtr[m+1]
break;
l = m + 1
else:
if m == 0: if m == 0:
sptr = self.__headerPtr[m] sptr = self.__headerPtr[m]
eptr = self.__headerPtr[m+1] eptr = self.__headerPtr[m+1]
break; break
elif ip > self.__headerSip[m-1]: elif ip > self.__headerSip[m-1]:
sptr = self.__headerPtr[m-1] sptr = self.__headerPtr[m-1]
eptr = self.__headerPtr[m] eptr = self.__headerPtr[m]
break; break
h = m - 1 h = m - 1
else:
if m == self.__headerLen - 1:
sptr = self.__headerPtr[m-1]
eptr = self.__headerPtr[m]
break
elif ip <= self.__headerSip[m+1]:
sptr = self.__headerPtr[m]
eptr = self.__headerPtr[m+1]
break
l = m + 1
if sptr == 0: return "N1" if sptr == 0: raise Exception("Index pointer not found")
indexLen = eptr - sptr indexLen = eptr - sptr
self.__f.seek(sptr) self.__f.seek(sptr)
b = self.__f.read(indexLen + 12) index = self.__f.read(indexLen + self.__INDEX_BLOCK_LENGTH)
l, h, mixPtr = (0, int(indexLen/12), 0) l, h, dataPrt = (0, int(indexLen/self.__INDEX_BLOCK_LENGTH), 0)
while l <= h: while l <= h:
m = int((l+h)/2) m = int((l+h) >> 1)
offset = m * 12 offset = int(m * self.__INDEX_BLOCK_LENGTH)
sip = self.getLong(index, offset)
if ip >= self.getLong(b, offset): if ip < sip:
if ip > self.getLong(b, offset+4):
l = m + 1
else:
mixPtr = self.getLong(b, offset+8)
break;
else:
h = m - 1 h = m - 1
else:
eip = self.getLong(index, offset+4)
if ip > eip:
l = m + 1;
else:
dataPrt = self.getLong(index, offset+8)
break
if mixPtr == 0: return "N2" if dataPrt == 0: raise Exception("Data pointer not found")
return self.returnData(mixPtr) return self.returnData(dataPrt)
def initDatabase(self, dbfile): def initDatabase(self, dbfile):
""" """
@ -191,17 +183,17 @@ class Ip2Region(object):
""" """
try: try:
self.__f = io.open(dbfile, "rb") self.__f = io.open(dbfile, "rb")
except IOError, e: except IOError as e:
print "[Error]: ", e print("[Error]: %s" % e)
sys.exit() sys.exit()
def returnData(self, dsptr): def returnData(self, dataPtr):
""" """
" get ip data from db file by data start ptr " get ip data from db file by data start ptr
" param: dsptr " param: dsptr
""" """
dataPtr = dsptr & 0x00FFFFFFL dataLen = (dataPtr >> 24) & 0xFF
dataLen = (dsptr >> 24) & 0xFF dataPtr = dataPtr & 0x00FFFFFF
self.__f.seek(dataPtr) self.__f.seek(dataPtr)
data = self.__f.read(dataLen) data = self.__f.read(dataLen)
@ -213,7 +205,6 @@ class Ip2Region(object):
def ip2long(self, ip): def ip2long(self, ip):
_ip = socket.inet_aton(ip) _ip = socket.inet_aton(ip)
return struct.unpack("!L", _ip)[0] return struct.unpack("!L", _ip)[0]
def isip(self, ip): def isip(self, ip):
@ -230,11 +221,12 @@ class Ip2Region(object):
def getLong(self, b, offset): def getLong(self, b, offset):
if len(b[offset:offset+4]) == 4: if len(b[offset:offset+4]) == 4:
return struct.unpack('I', b[offset:offset+4])[0] return struct.unpack('I', b[offset:offset+4])[0]
return 0 return 0
def close(self): def close(self):
self.__headerSip = None if self.__f != None:
self.__headerPtr = None
self.__f.close() self.__f.close()
self.__f = None
self.__dbBinStr = None
self.__headerPtr = None
self.__headerSip = None

View File

@ -6,72 +6,76 @@
" Date : 2015-11-06 " Date : 2015-11-06
""" """
import struct, sys, os, time import struct, sys, os, time
from platform import python_version
from ip2Region import Ip2Region from ip2Region import Ip2Region
def testSearch(): def testSearch():
""" """
" ip2region test function " ip2region test function
""" """
llen = len(sys.argv) argLen = len(sys.argv)
version = python_version()
algorithms = ["binary", "b-tree", "memory"]
if llen < 2: if argLen < 2:
print "Usage: python ip2Region.py [ip2region db file] [alrogrithm]" print("Usage: python testSearcher.py [ip2region db file] [alrogrithm]")
print "Algorithm: binary or b-tree" print("Algorithm: %s" % ", ".join(algorithms))
return 0 return 0
dbFile = sys.argv[1] dbFile = sys.argv[1]
method = 1
algorithm = "b-tree"
if (not os.path.isfile(dbFile)) or (not os.path.exists(dbFile)): if (not os.path.isfile(dbFile)) or (not os.path.exists(dbFile)):
print "[Error]: Specified db file is not exists." print("[Error]: Specified db file is not exists.")
return 0 return 0
if llen > 2: if argLen > 2:
algorithm = sys.argv[2] algorithm = sys.argv[2]
if algorithm == "binary": try:
method = 2 algorithms.index(algorithm)
elif algorithm == "memory": except Exception as e:
method = 3 algorithm = "b-tree"
print "initializing %s..." % (algorithm) print("initializing %s..." % (algorithm))
print "+----------------------------------+" print("+----------------------------------+")
print "| ip2region test program |" print("| ip2region test program |")
print "| Author: chenxin619315@gmail.com. |" print("| Author: chenxin619315@gmail.com. |")
print "| Type 'quit' to exit program |" print("| Type 'quit' to exit program |")
print "+----------------------------------+" print("+----------------------------------+")
searcher = Ip2Region(dbFile); searcher = Ip2Region(dbFile)
while True: while True:
if version[:1] == "2":
line = raw_input("ip2region>> ") line = raw_input("ip2region>> ")
else:
line = input("ip2region>> ")
line = line.strip() line = line.strip()
if line == "": if line == "":
print "[Error]: Invalid ip address." print("[Error]: Invalid ip address.")
continue continue
if line == "quit": if line == "quit":
print "[Info]: Thanks for your use, Bye." print("[Info]: Thanks for your use, Bye.")
break break
if not searcher.isip(line): if not searcher.isip(line):
print "[Error]: Invalid ip address." print("[Error]: Invalid ip address.")
continue continue
try:
sTime = time.time()*1000 sTime = time.time()*1000
if method == 1: if algorithm == "binary":
data = searcher.btreeSearch(line)
elif method == 2:
data = searcher.binarySearch(line) data = searcher.binarySearch(line)
else: elif algorithm == "memory":
data = searcher.memorySearch(line) data = searcher.memorySearch(line)
eTime = time.time() * 1000
if isinstance(data, dict):
print "%s|%s in %f millseconds" % (data["city_id"], data["region"], eTime-sTime)
else: else:
print "[Error]: ", data data = searcher.btreeSearch(line)
eTime = time.time()*1000
print("%s|%s in %5f millseconds" % (data["city_id"], data["region"].decode('utf-8'), eTime - sTime))
except Exception as e:
print("[Error]: %s" % e)
searcher.close() searcher.close()

17
binding/rust/Cargo.toml Normal file
View File

@ -0,0 +1,17 @@
[package]
name = "ip2region"
version = "0.2.0"
authors = ["biluohc <biluohc@qq.com>"]
include = ["./*", "../../data/ip2region.db", "../../Cargo.toml"]
[[bin]]
name="ip2region"
path="src/main.rs"
[dependencies]
lazy_static ={ version = "^1", optional = true }
[features]
lazy = ["lazy_static"]

View File

@ -0,0 +1,13 @@
[package]
name = "example"
version = "0.1.0"
authors = ["biluohc <biluohc@qq.com>"]
[dependencies]
[dependencies.ip2region]
path = "../"
# git = "https://github.com/lionsoul2014/ip2region"
# git = "https://github.com/biluohc/ip2region"
version = "*"
features = ["lazy"]

View File

@ -0,0 +1,114 @@
extern crate ip2region;
use ip2region::*;
use std::env;
use std::time::Instant;
// cargo run --release ../../../data/ip2region.db
static IPS: &'static [&'_ str] = &[
"117.136.105.202",
"47.95.47.253",
"127.0.0.1",
"10.0.0.1",
"1.1.1.1",
];
fn main() {
lazy();
overview();
}
fn lazy() {
for ip in IPS {
let start = Instant::now();
let res = memory_search(ip);
let end = start.elapsed().subsec_micros();
println!("lazy__ {:06} microseconds: {:?}", end, res);
let start = Instant::now();
let ip_addr = ip.parse().unwrap();
let res2 = memory_search_ip(&ip_addr);
let end = start.elapsed().subsec_micros();
println!("lazy__ {:06} microseconds: {:?}", end, res2);
if res.is_ok() && res2.is_ok() {
assert_eq!(res.unwrap(), res2.unwrap());
} else if res.is_err() && res2.is_err() {
} else {
panic!("not EQ")
}
}
}
fn overview() {
let args = env::args().skip(1).collect::<Vec<String>>();
let db_path = &args[0];
let mut ip2 = Ip2Region::new(db_path).unwrap();
let ip2o = ip2.to_owned().unwrap();
for ip in IPS {
// mem
let start = Instant::now();
let res = ip2o.memory_search(ip);
let end = start.elapsed().subsec_micros();
println!("memory {:06} microseconds: {:?}", end, res);
let start = Instant::now();
let ip_addr = ip.parse().unwrap();
let res2 = ip2o.memory_search_ip(&ip_addr);
let end = start.elapsed().subsec_micros();
println!("memory {:06} microseconds: {:?}", end, res2);
if res.is_ok() && res2.is_ok() {
assert_eq!(res.unwrap(), res2.unwrap());
} else if res.is_err() && res2.is_err() {
} else {
panic!("not EQ")
}
// binary
let start = Instant::now();
let res = ip2.binary_search(ip);
let end = start.elapsed().subsec_micros();
println!("binary {:06} microseconds: {:?}", end, res);
let start = Instant::now();
let ip_addr = ip.parse().unwrap();
let res2 = ip2.binary_search_ip(&ip_addr);
let end = start.elapsed().subsec_micros();
println!("binary {:06} microseconds: {:?}", end, res2);
if res.is_ok() && res2.is_ok() {
assert_eq!(res.unwrap(), res2.unwrap());
} else if res.is_err() && res2.is_err() {
} else {
panic!("not EQ")
}
// btree
let start = Instant::now();
let res = ip2.btree_search(ip);
let end = start.elapsed().subsec_micros();
println!("btree {:06} microseconds: {:?}", end, res);
let start = Instant::now();
let ip_addr = ip.parse().unwrap();
let res2 = ip2.btree_search_ip(&ip_addr);
let end = start.elapsed().subsec_micros();
println!("btree_ {:06} microseconds: {:?}", end, res2);
if res.is_ok() && res2.is_ok() {
assert_eq!(res.unwrap(), res2.unwrap());
} else if res.is_err() && res2.is_err() {
} else {
panic!("not EQ")
}
// \n
println!();
}
}

36
binding/rust/readme.md Normal file
View File

@ -0,0 +1,36 @@
## Rust 客户端
## 用法
Demo`example` 目录里
API文档 `cargo doc --features lazy --open` 可以看到所有。
运行测试: `cargo test --features lazy`
### 添加依赖
注意 `#` 是 toml格式的注释文件前三行的 `path``git` 是引用包的方式,`path` 是按路径引用包,`git` 是按git项目地址按实际情况选一个。
```toml
[dependencies.ip2region]
git = "https://github.com/lionsoul2014/ip2region"
# git = "https://github.com/biluohc/ip2region"
# path = "../"
version = "*"
# features = ["lazy"]
```
### 代码
查看 `example/src/main.rs`
### `lazy` feature 把 DB 直接打包进二进制
取消上面 toml 的 `# features = ["lazy"]` 行的注释即可使用,其 api 是 `memory_search``memory_search_ip`
只是目前 DB 足有3.2M,还是有些感人的。
关键的一行是 `features = ["lazy"]` ,不需要则可以注释或者删掉。

7
binding/rust/src/db.rs Normal file
View File

@ -0,0 +1,7 @@
pub const DB_PATH: &str = concat!(env!("CARGO_MANIFEST_DIR"), "/../../data/ip2region.db");
#[cfg(feature = "lazy")]
pub static DB_BYTES: &'static [u8] = include_bytes!(concat!(
env!("CARGO_MANIFEST_DIR"),
"/../../data/ip2region.db"
));

44
binding/rust/src/error.rs Normal file
View File

@ -0,0 +1,44 @@
use std::{self, io, net, str};
pub type Result<T> = std::result::Result<T, Error>;
#[derive(Debug)]
pub enum Error {
Io(io::Error),
Utf8(str::Utf8Error),
Addr(net::AddrParseError),
/// `224.0.0.0` ~ `239.255.255.255`
///
// `ff00::/8`
IpIsMulticast,
/// `0.0.0.0`
IpIsUnspecified,
/// `127.0.0.0/8`
IpIsLoopback,
///1. `10.0.0.0/8`
///
///2. `172.16.0.0/12`
///
///3. `192.168.0.0/16`
IpIsPrivate,
/// Unsupport Ipv6 Now
UnsupportIpv6,
NotFound,
}
impl From<io::Error> for Error {
fn from(e: io::Error) -> Self {
Error::Io(e)
}
}
impl From<str::Utf8Error> for Error {
fn from(e: str::Utf8Error) -> Self {
Error::Utf8(e)
}
}
impl From<net::AddrParseError> for Error {
fn from(e: net::AddrParseError) -> Self {
Error::Addr(e)
}
}

20
binding/rust/src/lazy.rs Normal file
View File

@ -0,0 +1,20 @@
use super::DB_BYTES;
lazy_static! {
static ref OWNED_IP_2_REGION: OwnedIp2Region = {
OwnedIp2Region {
db_bin_bytes: Cow::Borrowed(DB_BYTES),
first_index_ptr: get_u32(&DB_BYTES[..], 0),
total_blocks: (get_u32(&DB_BYTES[..], 4) - get_u32(&DB_BYTES[..], 0))
/ INDEX_BLOCK_LENGTH + 1,
}
};
}
pub fn memory_search<S: AsRef<str>>(ip_str: S) -> Result<IpInfo<'static>> {
OWNED_IP_2_REGION.memory_search(ip_str)
}
pub fn memory_search_ip(ip_addr: &IpAddr) -> Result<IpInfo> {
OWNED_IP_2_REGION.memory_search_ip(ip_addr)
}

375
binding/rust/src/lib.rs Normal file
View File

@ -0,0 +1,375 @@
#[cfg(feature = "lazy")]
#[macro_use]
extern crate lazy_static;
use std::cell::RefCell;
use std::fs::File;
use std::io::{self, Read, Seek, SeekFrom};
use std::net::IpAddr;
use std::{fmt, str};
mod db;
mod error;
pub use error::{Error, Result};
mod owned;
#[doc(hidden)]
pub use db::DB_PATH;
pub use owned::{OwnedIp2Region, OwnedIpInfo};
#[cfg(feature = "lazy")]
use db::DB_BYTES;
#[cfg(feature = "lazy")]
pub use owned::{memory_search, memory_search_ip};
const INDEX_BLOCK_LENGTH: u32 = 12;
const TOTAL_HEADER_LENGTH: usize = 8192;
thread_local!(static BUF: RefCell<[u8;256]> = RefCell::new([0;256]));
thread_local!(static BUF_BTREE: RefCell<[u8;TOTAL_HEADER_LENGTH]> = RefCell::new([0;TOTAL_HEADER_LENGTH]));
#[allow(non_snake_case)]
#[derive(Debug, Default, Clone, PartialEq)]
pub struct IpInfo<'a> {
pub city_id: u32,
pub country: &'a str,
pub region: &'a str,
pub province: &'a str,
pub city: &'a str,
pub ISP: &'a str,
}
impl<'a> IpInfo<'a> {
fn new(city_id: u32, fields: &[&'a str]) -> Self {
if fields.len() < 5 {
panic!(format!("invlid fields: {:?}", fields));
}
IpInfo {
country: fields[0],
region: fields[1],
province: fields[2],
city: fields[3],
ISP: fields[4],
city_id,
}
}
pub fn to_owned(&self) -> OwnedIpInfo {
OwnedIpInfo {
city_id: self.city_id,
country: self.country.to_owned(),
region: self.region.to_owned(),
province: self.province.to_owned(),
city: self.city.to_owned(),
ISP: self.ISP.to_owned(),
}
}
}
impl<'a> fmt::Display for IpInfo<'a> {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
write!(
f,
"{}|{}|{}|{}|{}|{}",
self.city_id, self.country, self.region, self.province, self.city, self.ISP
)
}
}
fn get_ip_info(city_id: u32, line: &[u8]) -> Result<IpInfo> {
let str = str::from_utf8(line)?;
let fields = str.split('|').collect::<Vec<&str>>();
Ok(IpInfo::new(city_id, &fields[..]))
}
fn get_u32(bytes: &[u8], offset: u32) -> u32 {
let offset = offset as usize;
let tmp = (bytes[offset] as i64) & 0x000000FF
| ((bytes[offset + 1] as i64) << 8) & 0x0000FF00
| ((bytes[offset + 2] as i64) << 16) & 0x00FF0000
| ((bytes[offset + 3] as i64) << 24) & 0xFF000000;
tmp as u32
}
fn ip2u32(ip: &IpAddr) -> Result<u32> {
if ip.is_ipv6() {
return Err(Error::UnsupportIpv6);
}
if ip.is_unspecified() {
return Err(Error::IpIsUnspecified);
}
if ip.is_loopback() {
return Err(Error::IpIsLoopback);
}
if ip.is_multicast() {
return Err(Error::IpIsMulticast);
}
match ip {
IpAddr::V4(v4) => {
if v4.is_private() {
return Err(Error::IpIsPrivate);
}
let mut sum: u32 = 0;
for (i, n) in v4.octets().iter().enumerate() {
sum += (*n as u32) << 24 - 8 * i;
}
return Ok(sum);
}
IpAddr::V6(_v6) => unreachable!(),
}
}
pub struct Ip2Region {
// db file handler
db_file: File,
//header block info
header_sip: Vec<u32>,
header_ptr: Vec<u32>,
header_len: u32,
// super block index info
first_index_ptr: u32,
last_index_ptr: u32,
total_blocks: u32,
}
impl Ip2Region {
pub fn new(path: &str) -> io::Result<Self> {
let file = File::open(path)?;
Ok(Ip2Region {
db_file: file,
header_sip: Vec::new(),
header_ptr: Vec::new(),
header_len: 0,
first_index_ptr: 0,
last_index_ptr: 0,
total_blocks: 0,
})
}
pub fn to_owned(&mut self) -> Result<OwnedIp2Region> {
OwnedIp2Region::new2(&mut self.db_file).map_err(Error::Io)
}
pub fn binary_search<S: AsRef<str>>(&mut self, ip_str: S) -> Result<OwnedIpInfo> {
let ip = ip_str.as_ref().parse::<IpAddr>()?;
self.binary_search_ip(&ip)
}
pub fn binary_search_ip(&mut self, ip_addr: &IpAddr) -> Result<OwnedIpInfo> {
BUF.with(|buf| {
let mut buf = buf.borrow_mut();
if self.total_blocks == 0 {
self.db_file.seek(SeekFrom::Start(0))?;
self.db_file.read_exact(&mut buf[..8])?;
self.first_index_ptr = get_u32(&buf[..8], 0);
self.last_index_ptr = get_u32(&buf[..8], 4);
self.total_blocks =
(self.last_index_ptr - self.first_index_ptr) / INDEX_BLOCK_LENGTH + 1;
}
let ip = ip2u32(ip_addr)?;
let mut h = self.total_blocks;
let (mut data_ptr, mut l) = (0u32, 0u32);
while l <= h {
let m = (l + h) >> 1;
let p = self.first_index_ptr + m * INDEX_BLOCK_LENGTH;
self.db_file.seek(SeekFrom::Start(p as u64))?;
self.db_file
.read_exact(&mut buf[0..INDEX_BLOCK_LENGTH as usize])?;
let sip = get_u32(&buf[..INDEX_BLOCK_LENGTH as usize], 0);
if ip < sip {
h = m - 1;
} else {
let eip = get_u32(&buf[..INDEX_BLOCK_LENGTH as usize], 4);
if ip > eip {
l = m + 1;
} else {
data_ptr = get_u32(&buf[..INDEX_BLOCK_LENGTH as usize], 8);
break;
}
}
}
if data_ptr == 0 {
Err(Error::NotFound)?;
}
let data_len = (data_ptr >> 24) & 0xff;
data_ptr = data_ptr & 0x00FFFFFF;
self.db_file.seek(SeekFrom::Start(data_ptr as u64))?;
self.db_file.read_exact(&mut buf[0..data_len as usize])?;
get_ip_info(
get_u32(&buf[..data_len as usize], 0),
&buf[4..data_len as usize],
).map(|i| i.to_owned())
})
}
pub fn btree_search<S: AsRef<str>>(&mut self, ip_str: S) -> Result<OwnedIpInfo> {
let ip = ip_str.as_ref().parse::<IpAddr>()?;
self.btree_search_ip(&ip)
}
pub fn btree_search_ip(&mut self, ip_addr: &IpAddr) -> Result<OwnedIpInfo> {
BUF_BTREE.with(|buf| {
let mut buf = buf.borrow_mut();
if self.header_len == 0 {
self.db_file.seek(SeekFrom::Start(8))?;
self.db_file.read_exact(&mut buf[0..TOTAL_HEADER_LENGTH])?;
let (mut i, mut idx) = (0, 0);
while i < TOTAL_HEADER_LENGTH {
let sip = get_u32(&buf[0..TOTAL_HEADER_LENGTH], i as u32);
let idx_ptr = get_u32(&buf[0..TOTAL_HEADER_LENGTH], i as u32 + 4);
if idx_ptr == 0 {
break;
}
self.header_sip.push(sip);
self.header_ptr.push(idx_ptr);
i += 8;
idx += 1;
}
self.header_len = idx
}
let ip = ip2u32(ip_addr)?;
let mut h = self.header_len;
let (mut sptr, mut eptr, mut l) = (0u32, 0u32, 0u32);
while l <= h {
let m = (l + h) >> 1;
if m < self.header_len {
if ip == self.header_sip[m as usize] {
if m > 0 {
sptr = self.header_ptr[m as usize - 1];
eptr = self.header_ptr[m as usize];
} else {
sptr = self.header_ptr[m as usize];
eptr = self.header_ptr[m as usize + 1];
}
break;
}
if ip < self.header_sip[m as usize] {
if m == 0 {
sptr = self.header_ptr[m as usize];
eptr = self.header_ptr[m as usize + 1];
break;
} else if ip > self.header_sip[m as usize - 1] {
sptr = self.header_ptr[m as usize - 1];
eptr = self.header_ptr[m as usize];
break;
}
h = m - 1
} else {
if m == self.header_len - 1 {
println!("m/hl: {}/{}", m, self.header_len);
sptr = self.header_ptr[m as usize - 1];
eptr = self.header_ptr[m as usize];
break;
} else if ip <= self.header_sip[m as usize + 1] {
sptr = self.header_ptr[m as usize];
eptr = self.header_ptr[m as usize + 1];
break;
}
l = m + 1
}
}
}
if sptr == 0 {
Err(Error::NotFound)?;
}
let block_len = eptr - sptr;
self.db_file.seek(SeekFrom::Start(sptr as u64))?;
let buf_size = (block_len + INDEX_BLOCK_LENGTH) as usize;
self.db_file.read_exact(&mut buf[..buf_size])?;
let mut data_ptr = 0;
h = block_len / INDEX_BLOCK_LENGTH;
l = 0;
while l <= h {
let m = (l + h) >> 1;
let p = m * INDEX_BLOCK_LENGTH;
let sip = get_u32(&buf[..buf_size], p);
if ip < sip {
h = m - 1;
} else {
let eip = get_u32(&buf[..buf_size], p + 4);
if ip > eip {
l = m + 1;
} else {
data_ptr = get_u32(&buf[..buf_size], p + 8);
break;
}
}
}
if data_ptr == 0 {
Err(Error::NotFound)?;
}
let data_len = (data_ptr >> 24) & 0xff;
data_ptr = data_ptr & 0x00FFFFFF;
self.db_file.seek(SeekFrom::Start(data_ptr as u64))?;
self.db_file.read_exact(&mut buf[0..data_len as usize])?;
get_ip_info(
get_u32(&buf[..data_len as usize], 0),
&buf[4..data_len as usize],
).map(|i| i.to_owned())
})
}
}
// cargo test --features lazy -- --nocapture
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn it_works() {
let mut ip2 = Ip2Region::new(DB_PATH).unwrap();
let ip2o = ip2.to_owned().unwrap();
for ip in vec![
"117.136.105.202",
"47.95.47.253",
"127.0.0.1",
"10.0.0.1",
"1.1.1.1",
] {
#[cfg(feature = "lazy")]
{
println!("lzay__: {:?}", memory_search(ip));
if ip2o.memory_search(ip).is_ok() {
assert_eq!(ip2o.memory_search(ip).unwrap(), memory_search(ip).unwrap());
} else {
assert!(memory_search(ip).is_err());
}
}
println!("memory: {:?}", ip2o.memory_search(ip));
println!("binary: {:?}", ip2.binary_search(ip));
println!("btree : {:?}", ip2.btree_search(ip));
if ip2o.memory_search(ip).is_ok() {
assert_eq!(
ip2o.memory_search(ip).unwrap().to_owned(),
ip2.binary_search(ip).unwrap()
);
assert_eq!(
ip2o.memory_search(ip).unwrap().to_owned(),
ip2.btree_search(ip).unwrap()
);
} else {
assert!(ip2.binary_search(ip).is_err());
assert!(ip2.btree_search(ip).is_err());
}
println!();
}
}
}

130
binding/rust/src/main.rs Normal file
View File

@ -0,0 +1,130 @@
extern crate ip2region;
use ip2region::*;
use std::env;
use std::io::{self, *};
use std::time::Instant;
// cargo run --release --features lazy
fn main() {
let args = env::args().skip(1).collect::<Vec<String>>();
println!(
r#"ip2region cli test
+-----------------------------------------------------------------+
| ip2region [db_file] [alrogrithm]
| format : [ip] [alrogrithm]'
| overview: cargo run --release"
| usage: cargo run --release -- [db_file] [alrogrithm]"
| exit: quit or exit or Ctrl+C
+-----------------------------------------------------------------+`"#
);
if !args.is_empty() {
let db_path = if args[0] != "." { &args[0] } else { DB_PATH };
let mut ip2 = Ip2Region::new(db_path).unwrap();
let ip2o = ip2.to_owned().unwrap();
let alg = if args.len() > 1 {
args[1].to_lowercase()
} else {
"memory".to_lowercase()
};
let mut buf = String::with_capacity(256);
loop {
buf.clear();
print!("ip2region>>");
io::stdout().flush().unwrap();
let line = match io::stdin().read_line(&mut buf) {
Ok(_) => buf.trim(),
Err(e) => panic!("[Fatal]: Read String from Stdin Error: {:?}", e),
};
if line.is_empty() {
continue;
}
if line == "quit" || line == "exit" {
println!("[Info]: Thanks for your use, Bye.");
break;
}
let ip_alg = line
.split_whitespace()
.filter(|s| !s.is_empty())
.collect::<Vec<&str>>();
let alg = if ip_alg.len() > 1 {
ip_alg[1].to_owned()
} else {
alg.clone()
};
let start = Instant::now();
let res = match alg.as_str() {
"memory" => ip2o.memory_search(ip_alg[0]).map(|o| o.to_owned()),
"binary" => ip2.binary_search(ip_alg[0]),
"btree" | "b-tree" => ip2.btree_search(ip_alg[0]),
miss => {
eprintln!("Not have the Algorithm: {:?}", miss);
continue;
}
};
let end = start.elapsed().subsec_micros();
match res {
Ok(i) => {
println!("[{:6} {:06} microseconds]: {}", alg, end, i);
}
Err(e) => {
eprintln!("[Error]: {:?}", e);
}
};
}
} else {
overview()
}
}
fn overview() {
let mut ip2 = Ip2Region::new(DB_PATH).unwrap();
let ip2o = ip2.to_owned().unwrap();
for ip in &[
"117.136.105.202",
"47.95.47.253",
"127.0.0.1",
"10.0.0.1",
"1.1.1.1",
] {
let start = Instant::now();
let res = ip2o.memory_search(ip);
let end = start.elapsed().subsec_micros();
println!("memory {:06} microseconds: {:?}", end, res);
#[cfg(feature = "lazy")]
{
let start = Instant::now();
let res = memory_search(ip);
let end = start.elapsed().subsec_micros();
println!("lazy__ {:06} microseconds: {:?}", end, res);
}
let start = Instant::now();
let res = ip2.binary_search(ip);
let end = start.elapsed().subsec_micros();
println!("binary {:06} microseconds: {:?}", end, res);
let start = Instant::now();
let res = ip2.btree_search(ip);
let end = start.elapsed().subsec_micros();
println!("btree {:06} microseconds: {:?}", end, res);
println!();
}
}

106
binding/rust/src/owned.rs Normal file
View File

@ -0,0 +1,106 @@
use super::*;
use std::borrow::Cow;
#[cfg(feature = "lazy")]
include!("lazy.rs");
#[allow(non_snake_case)]
#[derive(Debug, Default, Clone, PartialEq)]
pub struct OwnedIpInfo {
pub city_id: u32,
pub country: String,
pub region: String,
pub province: String,
pub city: String,
pub ISP: String,
}
impl OwnedIpInfo {
pub fn as_ref<'a>(&'a self) -> IpInfo<'a> {
IpInfo {
city_id: self.city_id,
country: &self.country,
region: &self.region,
province: &self.province,
city: &self.city,
ISP: &self.ISP,
}
}
}
impl fmt::Display for OwnedIpInfo {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
write!(f, "{}", self.as_ref())
}
}
pub struct OwnedIp2Region {
// super block index info
first_index_ptr: u32,
// last_index_ptr: u32,
total_blocks: u32,
db_bin_bytes: Cow<'static, [u8]>,
}
impl OwnedIp2Region {
pub fn new(path: &str) -> io::Result<Self> {
let mut file = File::open(path)?;
Self::new2(&mut file)
}
pub(crate) fn new2(file: &mut File) -> io::Result<Self> {
let file_size = file.metadata()?.len();
let mut bytes = Vec::with_capacity(file_size as usize);
file.read_to_end(&mut bytes)?;
let first_index_ptr = get_u32(&bytes[..], 0);
let last_index_ptr = get_u32(&bytes[..], 4);
let total_blocks = (last_index_ptr - first_index_ptr) / INDEX_BLOCK_LENGTH + 1;
let db_bin_bytes = Cow::Owned(bytes);
Ok(OwnedIp2Region {
first_index_ptr,
total_blocks,
db_bin_bytes,
})
}
pub fn memory_search<S: AsRef<str>>(&self, ip_str: S) -> Result<IpInfo> {
let ip = ip_str.as_ref().parse()?;
self.memory_search_ip(&ip)
}
pub fn memory_search_ip(&self, ip_addr: &IpAddr) -> Result<IpInfo> {
let ip = ip2u32(ip_addr)?;
let mut h = self.total_blocks;
let (mut data_ptr, mut l) = (0u32, 0u32);
while l <= h {
let m = (l + h) >> 1;
let p = self.first_index_ptr + m * INDEX_BLOCK_LENGTH;
let sip = get_u32(&self.db_bin_bytes[..], p);
if ip < sip {
h = m - 1;
} else {
let eip = get_u32(&self.db_bin_bytes[..], p + 4);
if ip > eip {
l = m + 1;
} else {
data_ptr = get_u32(&self.db_bin_bytes[..], p + 8);
break;
}
}
}
if data_ptr == 0 {
Err(Error::NotFound)?;
}
let data_len = (data_ptr >> 24) & 0xff;
data_ptr = data_ptr & 0x00FFFFFF;
get_ip_info(
get_u32(&self.db_bin_bytes[..], data_ptr),
&self.db_bin_bytes[(data_ptr + 4) as usize..(data_ptr + data_len) as usize],
)
}
}

File diff suppressed because it is too large Load Diff

Binary file not shown.

BIN
dbMaker-1.2.2.jar Normal file

Binary file not shown.