Query order balance via smart contract bypassing API

[saturn-network]

Using Saturn API simplifies a lot of the tricky bigint math that is essential in making Saturn Network exchange work. There are certain places where living without API is almost impossible, i.e. serving the order book for a token in under <100ms - you'd have to reimplement the API yourself. The API simply acts as a high-performance low-latency cache, and blockchain acts as the slow & secure source of truth for it.

Querying for the blockchain directly for the order balance, however, right before executing the order makes perfect sense. A good first contribution would be to modify this function to stop relying on the API and query the blockchain directly.

saturn.js/src/exchange/exchange.ts

Lines 217 to 223 in f06ac63

	private async verifyCapacity(amount: number, order: Order) {
	if (amount > Number(order.balance)) {
	throw new Error(`
	You attempted to trade more tokens (${amount}) than are available in the order (${order.balance}) for order_tx ${order.transaction}.
	`.trim())
	}
	}

Some helpful notes to get you started:

1. Review package.json of saturn.js and look at the scripts section. Hint: npm run build does something useful.
2. Learn to use yarn link or npm link to test drive your local copy of saturn.js with your scripts.
3. Some code snippets that you might find useful

let decimals = await token.decimals()
let rawOrderBalance = await exchange.remainingAmount(order.order_id)
let parsedOrderBalance = new Fraction(rawOrderBalance.toString()).div(new Fraction(10).pow(Number(decimals.toString())))

[saturn-network]

Bounty: 69,000 SATURN tokens of your choosing courtesy of Neuron.

[PseudoDeterminist]

The bad balance number seems to be coming from the ABI. Note the balance in the ABI query is:

"balance":"828.18209999034782608696"

When I try to sell into that using pricewatch-bot I get

You attempted to trade more tokens (828.1821) than are available in the order (828.18209999034782608696) for order_tx 0xcc4923e34e9ab99cf83fc88a1c98cf114d45760d9a0e8eca13c7206fae023d86.

I'm going to read up on how saturn.js can query the balance. I'll get it. No questions - just progress.

curl https://ticker.saturn.network/api/v2/orders/by_tx/ETC/0xcc4923e34e9ab99cf83fc88a1c98cf114d45760d9a0e8eca13c7206fae023d86.json
{"created_at":1557095984,"transaction":"0xcc4923e34e9ab99cf83fc88a1c98cf114d45760d9a0e8eca13c7206fae023d86","blockchain":"ETC","order_id":"3162","blocknumber":"7980452","contract":"0x3ec00ee8a4fbe81e7eea328029ce490654e8e11a","owner":"0x5860b14f9ac5c4ddaff8775a6c84b49a6c28dd07","active":true,"buytoken":{"address":"0xac55641cbb734bdf6510d1bbd62e240c2409040f","decimals":4,"blockchain":"ETC","name":"Saturn Classic DAO Token","symbol":"SATURN"},"selltoken":{"address":"0x0000000000000000000000000000000000000000","decimals":18,"blockchain":"ETC","name":"Classic Ether","symbol":"ETC"},"balance":"828.18209999034782608696","price":"0.000046","type":"BUY"}%

[PseudoDeterminist]

Confirmed (as I'm sure you have) that order is not tradeable on saturn.network or by bot. Also, attempted cheesy hack of saturn.js by modifying verifyCapacity as follows:

  if (amount > Number(order.balance)) {
  	amount = Number(order.balance)
  }

instead of throwing an error. This (I think) attempts to do the transaction as if the trader amount was containing the 9999's. In any case it stops the error condition so the trade can be attempted.

Then threw pricewatch-bot at it and got

Attempting to sell 828.1821 tokens
tx: 0xa1ea669529cd1abf76afe3f6ea7f677f8a0e4ca9e0fdc84054e7bbdea17832d5
Transaction 0xa1ea669529cd1abf76afe3f6ea7f677f8a0e4ca9e0fdc84054e7bbdea17832d5 on ETC failed.

That just updates progress report.

Next I'm still going to (do, not try!) what Neuron asked me to do, which is use saturn.js to query the balance, see if I can get rid of those ugly 9999's one way or another.

No questions.

[PseudoDeterminist]

Okay for completeness:

I cheesy-modified verifyCapacity in saturn.js as follows:

if (amount > Number(order.balance)) {
order.balance = amount
}

instead of throwing an error.

This (I think) attempts to sell into that order as if the order balance were in fact 828.1821. Although I'm not certain about that, as this was a noob cheesy hack.

In any case the transaction fails again, with a bad instruction error showing on blockscout:

https://blockscout.com/etc/mainnet/tx/0x6b915cbf07ec1a3b49fa1db3a9461a28f99f118e625779a1d9dfeb118cc031c3/internal_transactions

None of these three things are what Neuron asked me to do, and all of them are cheesy hacks, but it was my first day, and I wanted to know what these would do.

Still working on using saturn.js and the given code snippets to query the balance that way.

No questions.

[PseudoDeterminist]

Okay this is where I've been spinning my wheels for days: I added the following code (between // lines) to newERC223Trade:

  private async newERC223Trade(tokenAddress: string, amount: number, order: Order) : Promise<string> {
    console.log('--> newERC223Trade')
    let token = new Contract(tokenAddress, erc223, this.wallet)
//
    let decimals = await token.decimals()
    let exchange = new Contract(order.contract, this.exchangeAbi as any, this.wallet)
    let rawOrderBalance = await exchange.remainingAmount(order.order_id)
    let parsedOrderBalance = new Fraction(rawOrderBalance.toString()).div(new fraction(10).pow(Number(decimals.toString())))
    console.log(Number(rawOrderBalance), Number(parsedOrderBalance))
//

Basically all I want to do is query that rawOrderBalance and log it on the console!

What I get I cannot make sense of. Logged as objects, I get

rawOrderBalance: BigNumber { _hex: '0x056a1b3773be983c' }
parsedOrderBalance:  { s: 1, n: 24384640168749444, d: 625 }

Logged as a string or number, I get

rawOrderBalance: 474999998400000000
parsedOrderBalance: 1956513536

And this is for an order with no rounding problem, balance of: 9895.8333

The sale goes through in pricewatch-bot just fine for this order. I can query order.balance just fine but I think that comes from the SaturnAPI web3 interface.

rawOrderBalance is stumping me.

[PseudoDeterminist]

One thing I've learned from this adventure is funds are definitely safu. I see that pricewatch-bot and saturn.network interface trade correctly down to the smallest Saturn unit. Some bug in the beta release of market-maker-bot is allowing it to post weird problematic balances onto the exchange, but I imagine that doesn't worry you guys as to how to fix! I imagine that can also be prevented in SaturnAPI pretty easily, both as an error protection measure and to stop a potential attack on the exchange.

I already respected you guys highly, but having spent a few days trying to dig into your code, the respect has only gone way up. I couldn't have picked a better project to try my hand at.

I'd like to keep doing this. I know I can't waste your time asking too many questions. I'll keep studying anyway.

[saturn-network]

Notice that the exchange smart contract returns balance in selltokens. That is, the balance is denominated in collateral. If it is a buy order, then the returned balance is denominated in ether. If it is a sell order then the returned balance is denominated in tokens.

Saturn API always returns the balance of the order converted to tokens.

And this is for an order with no rounding problem, balance of: 9895.8333

Not surprised - newERC223Trade is being called against a buy order, so you get balance denomicated in ether and then divide it by token decimals.

Highly suggest you work on modifying verifyCapacity function that gets called in the right place as opposed to changing newERC223Trade that needs no alterations.

[PseudoDeterminist]

Unbelievable that my unstoppable and brilliant mind didn't think to check if the balance might be in ethers. Thanks for rescuing me...

First things first: Both of the orders for .0001 Saturn Classic belong to my wallets. I'll cancel them if you want them cleared. Or you can sell into them if you want, or whatever, using your tricks. I'd kinda like to do that myself, but I get that I'm taking a long time learning.

Second, (code shown at bottom of post) I queried the rawOrderBalance and got:

rawOrderBalance 4699991100 parsedOrderBalance 469999.11
amount: 0.0001

amount is how much saturn.js thinks we are trying to sell in Saturn tokens. it's correct, of course. but it doesn't agree with the 'order' object in saturn.js (queried from verifyCapacity):

{ created_at: 1557207769,
  transaction:
   '0x16687b936e556254a5a8cf99163fdd8339532d895aac6c85c69e8a52264b7731',
  blockchain: 'ETC',
  order_id: '3184',
  blocknumber: '7988220',
  contract: '0x3ec00ee8a4fbe81e7eea328029ce490654e8e11a',
  owner: '0xbad82c4735546a1f8a032ffd000cc46a9b8fa31a',
  active: true,
  buytoken:
   { address: '0xac55641cbb734bdf6510d1bbd62e240c2409040f',
     decimals: 4,
     blockchain: 'ETC',
     name: 'Saturn Classic DAO Token',
     symbol: 'SATURN' },
  selltoken:
   { address: '0x0000000000000000000000000000000000000000',
     decimals: 18,
     blockchain: 'ETC',
     name: 'Classic Ether',
     symbol: 'ETC' },
  balance: '0.00009999981063829787',
  price: '0.000047',
  type: 'BUY' }

what does agree with order.balance is rawOrderBalance (in ethers) times the sell price. But that doesn't agree with 'amount', it's off by tiny bit.

rawOrderBalance tells me that to sell into this order, (I think) I need to send exactly this much ethers: 0.0000000046999911, to the correct contract call. I don't know how do do that yet. But I think that might work!

One thing I tried was to just approve the .0001 amount in verifyCapacity (remove the if condition!)
-->transaction is sent, but fails with a bad instruction.
Note that the above amount (.0001 Saturn) times the price (0.000047) gives:
0.0000000047 ethers. So it will try to send tiny bit more than rawOrderBalance. I think this causes the fail?

I realize the task here is to just query the order balance directly. I've done that. But I really have no code to pull request yet... just used modified verifyCapacity with added code between // lines. When I send a pull request, I'm hoping it will actually do something with the query? Like even succeed in selling an order like this one?

My code modification in verifyCapacity is between the // lines:

private async verifyCapacity(amount: number, order: Order) {
      console.log('--> verifyCapacity')
//
    console.log(order)
      let tokenAddress = order.buytoken.address
      let token = new Contract(tokenAddress, erc223 as FunctionFragment[], this.wallet)
      let decimals = await token.decimals()
      let exchange = new Contract(order.contract, this.exchangeAbi as any, this.wallet)
      let rawOrderBalance = await exchange.remainingAmount(order.order_id)
      let parsedOrderBalance = new Fraction(rawOrderBalance.toString()).div(new Fraction(10).pow(Number(decimals.toString())))
      console.log('rawOrderBalance', Number(rawOrderBalance), 'parsedOrderBalance', Number(parsedOrderBalance))
      console.log('amount:', amount)
//
//    if (amount > Number(order.balance)) {
//      throw new Error(`
//        You attempted to trade more tokens (${amount}) than are available in the order (${order.balance}) for order_tx ${order.tran\
saction}.
//      `.trim())
//    }
  }

I don't know how to send the right amount of ethers (yet), though I know how many to send!
I don't think we can send less than 0.0001 Saturn as the 'amount' to saturn.js

But one or the other of those is what this order would need.

[PseudoDeterminist]

Oops I said send ethers to a Saturn buy order to clear it. That's dumb.
So yeah, you'd have to send .0001 Saturn... and we tried that.
Now what?

[saturn-network]

So it will try to send tiny bit more than rawOrderBalance. I think this causes the fail?

Yes

[saturn-network]

sending less than 0.0001 Saturn is impossible. Sending less Ether is definitely possible.

In terms of your contribution, cleaning up verifyCapacity is a good idea. As far as use case with the bots, changes will need to be made there. The goal of saturn.js is to simply be boring pipes that take you commands and execute them or give good error messages if you are doing something wrong.

Business logic should live elsewhere, i.e. in the bot code. After making the change in this library you might want to attempt to fixup this method so it returns correct results even with tiny amounts.

https://github.com/saturn-network/pricewatch-bot/blob/4136d35557cac59337e2ca5a87742158194d2657/pricewatch-bot.js#L89-L103

[PseudoDeterminist]

I left an order of 'balance' 82.387399999802125 to give my pending pull request something to chew on. Obviously you can easily clear it, but if you could leave it for a day I'll have time to get back to this tonight. Thanks!

[PseudoDeterminist]

Good grief I don't know how (and I'm not asking!) to divide two freaking numbers using these data structures. I know how to hang my program just by asking for wei divided by 10^etherDecimals.

If I take a long time to do a simple task, it's because every step I take involves learning a new javascript package. Great stuff! But all of it is new to me.

No questions.

edit -- I'm starting to learn the value of the functions Number() and toSuitableBigNumber()
edit 2 -- I'm literally just wandering through all this code and staring at it in awe. Trying to absorb its meaning by bouncing around in it enough. Wasting time! But wow. Why am I doing this? Because I want to do a simple operation that I don't know syntax or data structure for, and go looking... usually I can find it but I do get sidetracked on the way! Right now I want to get pricewatch-bot to sell into my order even though it's not the highest buy order. So, off on a little trek to understand how all that works.

[saturn-network]

ethers.js has a horrible BigNumber library that the user is forced to use... toSuitableBigNumber would ideally never exist. It is a crutch that lets us connect the rest of the program with ethers.js

bignumber.js used in bots is a much friendlier library